DOCUMENT RESUME 



ED 426 525 



EC 306 958 



AUTHOR 

TITLE 



INSTITUTION 

SPONS AGENCY 

PUB DATE 
NOTE 

CONTRACT 
AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Liberton, Cindy, Ed.; Kutash, Krista, Ed.; Friedman, Robert 
M., Ed. 

A System of Care for Children's Mental Health: Expanding the 
Research Base. Proceedings of the Annual Research Conference 
(8th, Tampa, FL, 1996) . 

University of South Florida, Tampa. Research and Training 
Center for Children's Mental Health. 

National Inst, on Disability and Rehabilitation Research 
(ED/OSERS) , Washington, DC. 

1996-00-00 

462p.; The twelve "chapters" of the proceedings have been 
separately analyzed, see EC 306 959-970. 

H133B90004 
Web site: 

http : //lumpy. fmhi . usf . edu/CFSroot/ rtc/proceeding8th 
Collected Works - Proceedings (021) 

MF01/PC19 Plus Postage . 

Adolescents; Child Welfare; Children; *Delivery Systems; 
Early Intervention; * Emotional Disturbances; Family 
Involvement; Financial Support; Information Systems; 

Juvenile Justice; *Mental Disorders; *Mental Health; Models; 
Program Evaluation; Research and Development; Research 
Methodology; *Systems Approach; Theory Practice Relationship 
Case Management 



ABSTRACT 



This document presents the proceedings of the 8th Annual 
(1996) Children's Mental Health Conference. Eleven main sections focus on the 
following topics: (1) evaluation efforts within states and systems of care; 

(2) financing strategies of systems of care; (3) family participation; (4) 
school-based services; (5) the child welfare system; (6) the juvenile justice 
system; (7) case management services; (8) models to improve service delivery; 
(9) early intervention approaches; (10) research methodology and information 
systems; and (11) utilization research. Also included are two lectures in the 
Gwen R. Iding Brogden Distinguished Lecture Series: "New Approaches to 
Evaluating Systems" (Heather B. Weiss) and "Outcome Accountability and System 
Reform: What Should They Mean in Policy and Practice?" (Charles Bruner) . (DB) 



******************************************************************************** 



* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

******************************************************************************** 




CS 

in 

VO 

CN 



s 



A System of Care for Children’s Mental Health: 
Expanding the Research Base. 
Proceedings of the Annual Research Conference 
(8th, Tampa, FL 1996). 



Cindy Liberton, ed. 
Krista Kutash, ed. 
Robert M. Friedman, ed. 



ijn 

cr 

'O 

cH 

O 

' 'i 

ERIC 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

Z CATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

his document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 




Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



2 



5C 



SthProcedingsCover 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8thtoc.html 



8 t h A NNU A t R ES e A fi CH CON P £ R E NC E . 1 9-9 S: 

A SYSTEM OF CARE FOR CHILDREN'S WlENTAl HEALTH: 

_ EXPANDING THE RESEARCH BASE 







ffiffiiual 







' -V 

' ' I 



/4 : 



mmsMm by ; 

THE RESEAltCH AND TRAINLNG CEh^ER ' 

FOR cHrtDREN's memtai: HEALTHU^ < : 






'■:f 



Edited by Cindy Liberton , Krista Kutash and Robert M. Friedman 

Table of Contents ^ Authors List^ Subiect Index 
RTC Home Paee -» CFS Home Page^Other FMHl Sites 4- USF 



8th Annual Children’s Mental Health Research Conference 

Proceedings 



CONTENTS 



® Chapter 1 - Evaluation Efforts within States & Systems of Care 

^ Chapter 2 - Financing Strategies of Systems of Care 

® Chajpter_3 - Family Participation 

^ Chapter 4 - School-based Services 

^ Chapter 5 - The Child Welfare System 

® Chapter 6 - The Juvenile Justice System 

^ Chapter 7 - Case Management Services 

^ Chapter 8 - Models to Improve Service Delivery 

^ Chapter 9 - Early Intervention Approaches 

^ Chapter 10 - Research Methodology & Information Systems 

^ Chapter 11 - Utilization Research 

^ The Gwen R.Iding Brogden Distinguished Lecture Series 



This is a publication of the Research and Training Center for Children’s Mental Health which is funded by the National 
Institute on Disability and Rehabilitation Research of the U.S. Department of Education under grant number H133B90004. 
The opinions contained in this publication are those of the grantee and do not necessarily reflect those of the U.S. 
Department of Education. 



io Back 



ERLC 



3 



12/10/98 10:50 AM 



Chapters 



http://lumpy.fnihi.usf.edu/CFSroot/rtc/proceeding8th/8thconts.html 



IHomei iiiaearcni iindexi 

8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

Table of Contents 

8th Annual Research Conference Proceedings 
Chapters:#1^2^3^445^6^748^9^10^11 

♦Gwen Iding Lecture Series 



♦ Go to Authors Index 

♦ Child & Family Studies ♦ CFS Divisions ♦ Faculty and Staff 
♦ ResourceNet ♦ Other FMHI Sites ♦ University of South Florida 



Chapter 1 

Focus on Evaluation Efforts within States and Systems of Care 



• An Overview of State Public Mental Health Programs for Children with a Serious Emotional 
Disturbance fSEDI 

• A Developmental Analysis of Neighborhood Governance: A Step In The Annie E. Casev Mental 
Health Initiative For Urban Children's Evaluation 

• Promoting Evaluation of Systems of Care: Stakeholder Needs & Technical Assistance Strategies 

• The Comprehensive Service Act: First Year Implementation Assessment 

• Access Vermont: A Statewide Evaluation of Non-categorical Service Delivery to Children and 
Families-Meeting Community Needs, Measuring Outcomes. 

• Conceptualization and Initial Development of the Ohio Scales 

• Impact of the Child and Adolescent Service System Program in California 

• Evaluation of the California Implementation of the Child and Adolescent Service System 
Program: A Review of Documents 

• Issues in Implementing a Randomly Assigned Study Design in a Community Setting: Lessons 
from the Vanderbilt/Stark County Evaluation Project 

• Evolutionary Changes in a Local System of Care for Severely Emotionally Disturbed Children 
and Adolescents: The Pendulum is Swinging 

♦ Return to Chapter List ♦ Go to Authors List jy] Go to Top of Page 



Chapter 2 

Focus on Financing Strategies for Systems of Care 



o 

ERIC 



Arizona's Implementation of Managed Care in the Public Systems: A State's Perspective 
The Effect of Intermediate Services On the Costs of Treating Children & Adolescents With 
Emotional and Behavioral Problems: A Preliminary Inquiry 
Do Intermediate Services Save Money? A System-Level Perspective 

The Impact of Managed Mental Health Care on Child and Adolescent Dispositions in a Regional 
Emergency Mental Health Service 

The Cost of Residential Schools for Children with Severe Emotional Disturbance 
A Comparison of Children and Adolescents Funded by Medicaid and Commercial Insurance in an 
Inpatient Psychiatric Facility 



4 



12/10/98 10:51 AM 



Chapters 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8thconts.html 



♦ Return to Chapter List ♦ Go to Authors List 'jgj Go to Top of Page 



Chapter 3 

Focus on Family Participation 



• Measuring Satisfaction with Community-Based Services for Children with Severe Emotional 
Disturbances: A Comparison of Questionnaires for Children and Parents 

• Use of Consumer Satisfaction Information to Promote Change in Child/ Adolecsent Mental 
HealthSettings 

• Parent Satisfaction with Mental Health Staff Interactions: The Development. Validity, and 
Reliability of The Parent Satisfaction Scale and The Parent-Staff Interaction Scale 

• Measuring Perceptions of Family Involvement in Service Provision for Youth with Serious 
Emotional Disturbance 

• Families' Perception of the Effects of Respite 

• Parent-Professional Collaboration: A Pilot Study of Parents as Researchers 

• The Significance of V61.xx - "Family Problems" in Mental Health Care for Children 

• The Influence of Family Functioning on the Incidence of Conduct Disorder Among Children and 
Adolescents 



♦ Return to Chapter List ♦Go to Authors List jjgj Go to Top of Page 



Chapter 4 

Focus on School-based Services 



• Facilitating the Social/Emotional Development of Middle School Students: A Model for 
Improving School-Based Collaboration 

• School-Based Wraparound: How Implementation and Evaluation Can Lead to System Change 

• Identification of Students with SEP: Correlates of State Child-Count Data 

• Comparison of Children and Adolescents with Serious Emotional Disturbance Served in Hospital 
and in School 

• One Year Outcome Findings of the Vanderbilt School-Based Counseling Evaluation Project 

• Interagency Collaboration Efforts with Families with Severe School Refusal Problems 

• Day Treatment for Children with Emotionally and Behavior Disorders: A Program Evaluation 

• Designing. Implementing, and Evaluating a School-Based Psvchoeducational Group for Children 
With Behavioral Problems from Families with Substance Abuse Issues 



♦ Return to Chapter List ♦Go to Authors List g| Go to Top of Page 



Chapter 5 

Focus on the Child Welfare System 



• A Controlled Study of Foster Children with Emotional/Behavioral Disturbances: Equivalence of 




5 



12/10/98 10:51 AM 



Chapters 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8thconts.html 



Groups Across Characteristics 

• Impact of an Experimental Foster Care Intervention on Emotional & Behavioral Adjustment: 
Examination of Preliminary Outcomes Using Growth Curve Analysis 

• Examining the Nature of the Independent Variable In a Controlled. Wraparound Foster Care 
Study: What Worked and Whv? 

• The Characteristics of Urban Children and Families Served by Child Welfare Agencies: The 
Satellite Family Outreach Program 

• Mental Health Screening in Foster Care: A Model for Community-Based Service Delivery and 
Research in Baltimore 



♦ Return to Chapter List ♦ Go to Authors List« [y[ Go to Top of Page 



Chapter 6 

Focus on the Juvenile Justice System 



• PeliiiQuencv and Mental Illness: The Intersection of Problems and Systems 

• Assessing the Mental Health of Adolescents in the Mental Health and Juvenile Justice Systems 

• Psychopathology and Symptomatology in Incarcerated versus Hospitalized Youth 

• Boot Camp: Is It Working? 

♦ Return to Chapter List ♦ Go to Authors List| «|| j Go to Top of Page 



Chapter 7 

Focus on Case Management Services 



• Providing Intensive Child Case Management Services: What Do Case Managers Do With Their 
Time? 

• Outcome-Oriented Case Planning in Child and Family Services 

• Systems Outcomes from Case Management for Children with Serious Emotional Disturbance 

• Child and Family Outcomes from Intensive Case Management for Children with Emotional 
Disturbance in New York State 

• Project Seed: Services Evaluation and Dissemination 

• Effectiveness of Intensive Case Management for Homeless Adolescents after Twelve Months 

♦ Return to Chapter List ♦ Go to Authors List| g1 Go to Top of Page 



Chapter 8 

Focus on Models to Improve Service Delivery 



• Symposium: Alternatives to the Hospitalization of Youth Presenting Psychiatric Crises 

o Empirical Evaluation of an Alternative to Hospitalization for Youth Presenting Psychiatric 
Emergencies 

o An Experimental Study of the Outcomes Associated with Three Crisis Intervention 



O 

ERIC 



6 



12/10/98 10:51 AM 



Chapters 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8thconts.html 



Programs for Children in Psychiatric Crisis and Their Families: Preliminary Findings 
o The Alternatives to Residential Treatment Study: Initial Findings 



• Coming Home: The Full Inclusion of Children with Serious Emotional Disturbances 

• Challenges to the Use of Respite for Children Receiving Short-term In-home Psychiatric 
Emergency Services 

• Mobile Outreach for Children and Families: An Effective Approach to Stabilization 

• A Study of Inpatient Treatment in the 9Q's 

• Symposium: Transitional Youth: The Challenges and Service Needs of Youth with Serious 
Emotional Disturbances as Young Adults 

• Adolescent Suicide: Implications of Coping. Family Functioning and Their Interactions for 
Prevention and Intervention 

• Gender Appropriate Services For Adolescents with Serious Emotional Disturbances: A Gender 
Competency Model 

• Multicultural Mental Health Training Program: A Comprehensive Training Program for 
Improving Service Delivery to Ethnic Minority Children and Families 

♦ Return to Chapter List ♦ Go to Authors Listl yj Go to Top of Page 



Chapter 9 

Focus on Early Intervention Approaches 



• Child Maltreatment and Prenatal Exposure to Alcohol and Other Drugs: Some Preliminary 
Findings 

• Evaluation of the Total Family Strategy Program 

• Public Mental Health and Head Start are Partners in the Ventura County System of Care 

♦ Return to Chapter List ♦ Go to Authors List^l Go to Top of Page 



Chapter 10 

Focus on Research Methodology and Information Systems 



• Parental Agreement on Ratings of Child Behavior: Measures of Agreement and Related Factors 

• CAFAS as a Measure of Child and Family Functioning Among Children and Families Enrolled in 
Intensive Case Management in New York State 

• The CAFAS as an Outcome Measure in the Alternatives to Residential Treatment Study 

• An Innovative Methodology for Assessing Children's Mental Health Services Needs 

• Developing an Effective, Standardized Client Infomiation System for Child Mental Flealth 
Treatment Centers 

• Enchancing the Methodology of Social Validation: A Survey of Child and Family Services Issues 

• Making "Cents" of Outcome Measures in a Partial Hospitalization Setting 

• A Functional Integrator In Systems of Care 

• An Alternative Approach to the Study of the Children's Mental Flealth Service System: 
Anthropological Models 

♦ Return to Chapter List ♦ Go to Authors List|^ Go to Top of Page 

O 

ERLC 



12/10/98 10:51 AM 



Chapters 



http://lumpy.frnhi.usf.edu/CFSroot/rtc/proceeding8th/8thconts.html 



Chapter 11 

Focus on Utilization Research 



• Infomiation Equity: A Critical Component of Strong Service Systems 

• Utilization of Children's Mental Health Services: Differentiating Families Who Use 
Community-Based Services From Those Who Do Not 

• Children's Mental Health Needs and Services in an Urban Area: Mapping and Analysis 

• Addressing the Barriers to Mental Health Services for Inner-City Children and their Caretakers 

• Referral Bias?: A Clinical Comparison of African-American and Caucasian Children Referred to a 
Child Psychiatric Outpatient Clinic 



♦ Return to Chapter List # 00 to Authors 




The Gwen R. Iding Brogden Distinquished Lecture Series 



• New Approaches to Evaluating Systems 



Heather B. Weiss, Ed.D. 



• Outcome Accountability and System Reform: What Should They Mean in Policy and Practice? 



Charles Brunner: Comments by Mary Teleford, & Cliff Davis 

♦ Return to Chapter List ♦ Go to Authors List |y j Go to Ton of Page 



ERIC 



8 



12/10/98 10:51 AM 



8th.3 



http://lumpy.fmhi.usf.edU/CFSroot/rtc/proceeding8th/8th.3.html 



I Home 1 1 search 1 1 index! 

8th Annual Research &, Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

An Overview of State Public Mental Health Programs for 
Children with a Serious Emotional Disturbance 

Authors 



Introduction Methods Results References 
Return to Table of Contents 



Introduction 

Over the past decade, increasing concern has been expressed about the mental health of our children. 
This has been caused by a number of factors including widespread family breakdown, the growing 
number of children bom to teen age mothers, the increasing rates of substance abuse and school 
dropouts. Many of these children receive no care and of those that do, many receive it from state and 
local public mental health agencies. Thus, it is important to examine how well the states are addressing 
this need. 
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Methods 

The data for this report come from the individual State Federal Block Grant Plan and Implementation 
Reports. The Federal State Block Grant legislation. Public Law (P.L.) 102-321, requires that the states 
develop overall program plans and annual progress reports. The plans must address twelve criteria 
pertaining to the development of community based systems of care. Not surprisingly, the quality of the 
plans and reports varies significantly across the states as does the amount of information provided. 
However, these reports have improved significantly due to better management information systems, 
better record keeping, and the participation of State Planning Councils 'with consumer and family 
representation. 
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Results 



Data from the State Federal Block Grant Plan and Implementation Reports suggest that all states have 
made progress in establishing an organized community based system of care for children and 
adolescents with serious emotional disturbances (SED) and their families. All 50 states report that they 
have adopted the Federal Child and Adolescent Service System (CASSP) principles for what constitutes 
an organized comprehensive community based system of care. These principles state that the systems 
should be comprehensive, coordinated, community based, family centered and culturally competent. 

In some states, this system development has been further enhanced by initiatives from the Governor, 
state legislators and local government officials. In other states, system development is impacted by 
litigation, especially in regard to inappropriate hospitalization. In still other states, state laws have been 
enacted which mandate that systems or system components such as local interagency coordinating 
councils be developed. 



Furthermore, all the states report efforts to develop interagency collaborations, usually with state and 
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local education, health, child welfare and juvenile justice agencies. Children's mental health service 
agencies are now linked with child welfare and social service agencies in 48 states, with educational 
agencies in 44 states, with juvenile justice agencies in 28 states and with health agencies in 28 states. 

Family Support 

There has been significant growth in family support and family preservation activities. In 1993, 

Congress passed legislation that, for the first time, made a significant investment in preventive services 
aimed at keeping families together. This legislation, known as the Family Support and Preservation Act, 
passed as part of the Omnibus Budget Reconciliation Act (OBRA) of 1993 (P.L. 103-66). As such, it 
was the first major new child welfare legislation in over a decade. In 1994, family preservation home 
builder programs were established in 35 states, and statutes addressing family preservation in 21 states. 

Concomitant with this, there has been tremendous growth in the movement to establish grass routes 
family support groups throughout the country. 

Serving Children with SED 

States reported serving an increased number of children with SED. The "most in need" target population 
included those persons under 18 years old who had a mental illness; that is, had a DSM-III R diagnosis, 
experienced serious functional impairment, had multi agency involvement, and whose illness duration 
has been greater than one year. Estimates of the prevalence of SED among children vary from 2-5% to 
12% of the child population. While there are varying estimates of need and demand for services, for the 
purpose of this report the most conservative estimates will be used; that is that 2% of all children under 
age 1 8 have a SED, and that 1 .2% of all children under 1 8 are estimated to need services for SED from 
the public sector at any given point in time. 

In an attempt to assess state programs in regard to how many children with SED they serve, these 
numbers were examined on a state-by-state basis. In examining these data, it was clear that there was 
great variation across the states in the numbers of children with SED served in comparison to the 
conservative estimates of target populations. The reports of percent of children with SED served ranged 
from 8 to 100%. Moreover, 14 states exceeded the proportion of children estimated to need services in 
the public sector in FY 1993-1994 (see Table 1). Another six states came close to these targets. These 
states included Alabama, Georgia, Indiana, Kentucky, Maine and South Dakota. 

Development of Community Based Services 

States also reported progress in developing community-based service arrays to substitute for inpatient 
care. According to the Block Grant Reports, the community based services for which there was the 
greatest demand and need were respite, crisis care and home-based services, as well as flexible 
wraparound services. Moreover, in order to further improve access, more and more states reported 
initiatives to provide services in schools and provide single points of service system entry. 

The major financing sources reported were private insurance and/or foundation money, Medicaid, and 
funds from state and local jurisdictions. At a national level, private sources funded about 44% of care, 
with Medicaid funding 19% and state and local jurisdictions funding 28%. During the 1980's Medicaid 
expenditures grew faster than private insurance expenditures. 

Medicaid eligibility procedures appear to have benefited state mental health programs in many ways. 
Requirements for pre-admission screening for hospitals may have prevented inappropriate admissions. 
Additionally, most states reported the ability to obtain Medicaid reimbursement for case management 
and rehabilitation services. Furthermore, many states have or are in the process of obtaining Medicaid 
waivers, despite the outcome of the national health care reform legislation. 

Use of Inpatient Hospital Services 

States report varying degrees of success in reducing use of hospital services, with admission rates 
ranging from 0 per 100,000 children, to over 100 per 100,000 children. The states with the highest rates 
and the ones with the lowest rates are displayed in Table 2. While some states reported success in 
reducing their hospital use, others have not been successful, and still others had mixed results. The states 
which report success number almost 25, and attribute the decrease in use to closing state hospitals and/or 
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reducing the number of beds for children, and developing alternative sources of care. However, seven 
states have experienced increasing rather than decreasing use of hospital care. Cormecticut, Idaho, 
Louisiana and Tennessee, report mixed results. For example, in Idaho, the number of admissions 
declined, but total number of bed days and average length of stay increased. Other mixed results 
included those states in which the past declines in use had leveled off and/or reversed direction over the 
past year. These data are presented in Table 3. 

To explain these varying degrees of success, the availability of alternative community services was 
examined, including the reported number of case managers and rate of screening for hospital admission, 
along with the number and array of alternative facilities, such as residential treatment centers (RTCs) 
and tiierapeutic foster and group homes. In California, for example, RTCs and groups homes were often 
used in lieu of hospitals. 

Case Management 

Reports indicated that Case Management was present in all states but Wyoming, which was in the 
process of developing this service. The percent of children with SED who received case management 
services ranged from about 8% to 100%. As can be seen in Table 4. 15% or fewer such children received 
case management in 1 1 states, whereas 50% or more of the children with SED received case 
management services in 10 other states. When this is compared to the conservative 1.2% estimate of 
children who need to use publicly funded services for SED, there appeared to be somewhat of an inverse 
relation between the states who reported the highest percent of children with SED served, and the 
proportion of those who receive case management. It appeared that the higher the proportion of children 
with SED who receive services, the lower the proportion who receive case management (Table 51. Of 
course, these data do not take into account differing case management models and client ratios. 

Programs for Homeless Children 

Programs for Homeless Children with SED were examined across the 50 states using block grant data 
along with data from the Program to Assist in the Transition from Homelessness (PATH). While all or 
almost all of the states report having programs to address the needs of homeless families and children, 
not all of them addressed homeless children with SED per se, and some did not specify numbers of such 
children served. Instead reference was made to services for homeless families and runaway youth in 
general. These data, however, did indicate that Arkansas, Maine, and North Dakota spent a sizable 
portion of their PATH money on children with SED. Other states such as Georgia, Kentucky and Oregon 
had conducted or were in the process of conducting surveys to assess the number of homeless children 
with SED. In addition, both Kentucky and Washington have received research demonstration grants 
targeted to providing services for homeless children with SED. 
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Introduction 

The Annie E. Casey Foundation's Mental Flealth Initiative for Urban Children is a five year, 
neighborhood scale program that seeks to demonstrate new ways of delivering comprehensive and 
coordinated, culturally appropriate, and family-sensitive mental health services to low income urban 
communities, and to work with states to improve the policies and practices supporting these services. 
Four urban neighborhoods in the cities of Ilichmond, Miami, Houston and Boston were selected to 
implement this initiative. 

The Florida Mental Health Institute was contracted to conduct the evaluation of the Initiative's outcomes 
in four areas: (1) overall quality of life in the neighborhood, (2) neighborhood governance, (3) interface 
between family needs and system response, and (4) documentation of movement of children through the 
system and the budget allocations associated with these moves. 

The emphasis on neighborhood governance is one of the unique aspects of the Mental Health Initiative 
for Urban Children which describes this as a mechanism for the delegation of real authority and 
decision-making from government to the neighborhood groups. This paper focuses on the outcomes of 
the process of neighborhood governance development including a description of the method of data 
collection, the creation of a developmental framework for data analysis, and a brief discussion of the 
findings. 
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Methodology 

Data collection for the neighborhood governance study involved two methods: a survey to establish the 
membership, structure, and composition of the neighborhood boards; and open-ended interviews with at 
least two respondents from each of the following three categories: (1) board members/neighborhood 
residents; (2) local level personnel from organizations, agencies or community associations; and (3) 
State agency representatives. 

Once the data were collected, a process involving review of the literature on community organizations, 
feedback from informal contacts with key informants at each site and formal reports from technical 
assistance staff working with the sites, and preliminary review of interview data 
led to the creation of a framework to describe governance board development. This framework depicts 
this development as moving along several attributes to effective administration and planning. Seven 
attributes were identified: (1) Community Information. (2) Community Involvement. (3) 
Consciousness/Knowledge. (4) Linkages. (5) Organization. (6) Programmatic Involvement, and (7) 
Resource Management Capacity (see attached defintitions). 
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Each of the attributes is divided into two or more aspects which define the attribute in more detail. For 
example, the attribute Consciousness/Knowledge is divided into four aspects: Belief in the Capacity to 
Change, Overall Vision of the Initiative, Knowledge of Human Service Systems, and Awareness of 
Political Process. The develop-mental stages for each aspect of an attribute include a continuum of nine 
stages. Stages 1, 3, 5, 7, and 9 have been defined. The intermediate stages 2, 4, 6, and 8 are not defined, 
but are included to allow for variance in the assignment of stages of development. 
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Discussion of Findings 

The analysis has revealed significant similarities across the four sites. The overall development of 
neighborhood governance in three of the sites that have established Boards falls between Stages 1 and 3 
across all developmental attributes. The East Little Havana site had not selected a Board at the time of 
the data collection for this report, and therefore, no developmental stages were assigned. The attributes 
of Consciousness/Knowledge, Organization, Community Involvement and Linkages have been selected 
for discussion in this paper. 

Consciousness/Knowledge 

A common element at all sites was the mixture of optimism and pessimism regarding the Board's 
capacity to influence the process of change in the system. Respondents at the state and local levels show 
varying degrees of optimism about the future of the Initiative. While they recognize the challenges that 
lie ahead, they also emphasize the accomplishments of having brought a diverse group of individuals 
together around this effort. 

At the neighborhood level, however, the resident representatives on the Board are not yet confident 
about their chances to effect meaningful and lasting changes in service systems. Their pessimism seems 
to be rooted in historical factors affecting the relationships between the state, city/county, and the 
community. In particular, residents expressed their disappointment at other initiatives that were 
implemented in the past that left the community untouched and their problems unsolved. A fear that this 
experience may be repeated once more, combined with a sense of fhistration over the lack of concrete 
services and programs at this point of the Initiative, are major challenges to the development of trust, a 
sense of partnership, and improved optimism among resident representatives on the Boards. 

Another interesting trend within this attribute is the presence of several visions of the Initiative which 
vary depending on whether the source is the neighborhood, the local level (e.g., city, county providers), 
or the state level. The neighborhood's vision seems to focus on short-term expectations 
in the area of service delivery and program development. Neighborhood residents see this Initiative as an 
opportunity to bring much needed services to the community, and upgrade existing programs to better 
serve children and families. 

The state level, on the other hand, envisions the Initiative as an opportunity to accomplish long-term 
system reform in the area of children's mental health. This level sees programmatic development more as 
a means to an end, rather than as the result of reform itself 

The local level seems to combine a short and long-term vision of the Initiative. Although, like the state, 
it recognizes system reform as the ultimate goal, like the neighborhood it sees the Initiative as an 
opportunity to supplement existing pools of money that are already being invested in the targeted 
communities (e.g., empowerment zones, Medicaid, full service schools, etc.). Another aspect of the local 
level vision is the perception of the Initiative as a chance to test the viability of new approaches to 
service delivery at the neighborhood level. 

Organization 

The most striking trend with regard to Board organization is the high degree of confusion across all sites 
regarding the roles of the different levels involved in the Initiative. Although in principle all parties 
agree that this is a community -driven effort, and that the neighborhood should be a key player and 
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driving force, in practice, there appear to be conflicting views of the degree to which the neighborhood 
should be in control. 

Residents perceive that their involvement should be at all levels of the decision-making process. They 
also express a need to catch up with the knowledge and expertise of the government level 
representatives, and feel fhistrated when they are not treated as equal partners. Local and state level 
officials, on the other hand, are concerned that the community does not have the necessary expertise and 
experience to control the entire process. 

An additional trend identified in the area of Organization relates to the distribution of labor. As a result 
of neighborhood representatives' need to feel included at all levels of the Initiative, the demands of 
serving on the Board become overwhelming. Tasks are highly concentrated on a few dedicated, 
politically aware volunteers who self-select themselves to be on the Board. These volunteers participate 
on several committees at a time, in addition to attending training events and Board meetings. 

A phenomenon related to the concentration of labor on a few experienced individuals with a history of 
community involvement is the influence of strong personalities on the Board and among the Initiative’s 
staff. During the initial phases of planning and implementation, all sites appear to have experienced this 
phenomenon to some extent, and dealt with it differently. In some cases, tihe identity of the Board came 
to be strongly tied to these individuals, leading other members to feel excluded from the process. 

Community Involvement 

Board representativeness is one of the biggest challenges facing all four Boards. The answer to the 
question of who are the Board's constituents, and therefore, to whom the Board is accountable remains 
elusive. Issues of representativeness cut across several lines including ethnicity, gender, age, 
immigration status, official status, provider status, parental status (e.g., parents of at-risk children), etc. 

The data analyzed for this report suggest that some of the difficulties experienced by the Boards with 
regard to representativeness may be related to the type of base upon which the Boards were founded. 
Boards that used pre-existing organizations reportedly inherited their perceived shortcomings. In Boston, 
the Healthy Boston Coalition's perceived inadequate inclusion of Latinos, other non- African American 
minorities, and parents of at-risk children targeted by the Initiative, is viewed by some as the cause of 
their struggle with representativeness. In Houston, concerns are related to the perceived over 
representation of providers on the Board. Representation challenges of Boards which started from 
scratch, such as Richmond and Miami, are linked to the need to broaden the existing resident 
representation on the Board. 

Linkages 

The main finding with regard to the linkages that the different levels of the Initiative have established 
among themselves is that sites appear to rely on a combination of direct and indirect links. Direct links 
are characterized by the state, local, provider and neighborhood levels communicating directly with each 
other in dyads (e.g., state-to-local, state-to-neighborhood, local-to-neighborhood, etc.^ The data suggest 
that this situation has led to the third party feeling excluded from the communication loop. A variation 
of direct links is the inclusion of state and local level representatives on the Board, as in the case of 
Boston and Miami. Indirect links refer to the use of intermediate channels of communication. This could 
be staff persons such as the State Coordinators and Project Director/Managers in Boston and Richmond, 
or an organization such as the Casey Policy Council in Houston, or the Children's Service Council in 
Miami. 

Of particular interest is the response elicited by the participation of providers in this Initiative. Except 
for the Houston Board which drew its original membership from a group of provider organizations, the 
other sites appear to have mixed feelings about any links with providers. These feelings are attributed by 
some to a fear that the Initiative would become provider-driven rather than neighborhood-driven. 
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The Developmental Framework for neighborhood governance is a significant first step toward 
describing the mechanisms and events diat lead to effective community participation within 
the context of systems reform. The framework's delineation of a tentative path toward neighborhood 
governance should provide a sounding board against which to challenge the proposed course and 
generate new ones. It is hoped that the repeated application of this framework to data generated during 
the remaining years of the neighborhood governance study will further the understanding of the complex 
process designed for this intervention. 
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Introduction 

With the fundamental reconceptualization of mental health care for children and their families (Knitzer, 
1993), and accelerating changes in service financing and delivery, there is increasing need for 
accountability and greater understanding of the impact of these systems changes (Saxe, Richmond, 
Gardner & Cross, 1994). Public sector agencies responsible for mental health services for children are 
facing increased expectations for evaluation, but often lack the resources, tools and experience. 

This paper summary describes a needs assessment conducted by the Technical Assistance (TA Center) 
for the Evaluation of Children's Mental Health Systems that was designed to provide information about 
the evaluation interests and technical assistance needs of three key stakeholder groups. The TA Center 
works with state and local communities to enhance their capacity to conduct and utilize evaluation of 
children's mental health system changes. The results of the needs assessment and a pilot site have guided 
the TA Center's approach to technical assistance strategies and activities. 

Return to Top 



Method 

Three stakeholder groups were included: (1) family members of children with serious emotional 
disorders, (2) state administrators of children's mental health programs, and (3) members of state 
legislatures and their staff. Information was gathered between April and September 1994. 

Family members contributed information through a focus group conducted at the conference held by the 
Research and Training Center on Family Support and Children's Mental Health in Portland, OR in April 
1994. Nine parents, identified through family support/advocacy organizations in their states, 
participated. The purpose of the focus group was to elicit from parents the crucial questions they have 
about service effectiveness and esponsiveness. Participants were also asked about how technical 
assistance might be used to increase families' involvement in evaluation. 

Twenty-three state government children's mental health administrators were interviewed between May 
and September 1994. The sample was drawn from the membership of the State Mental Health 
Representatives for Children and Youth (SMHRCY). Interviews were semi-structured, took between 45 
and 75 minutes to complete, and included queries about system changes, evaluation experience and 
resources, and technical assistance needs related to evaluation. 

State legislators were assembled for a focus group designed to explore their needs for information. The 
forum, held in Washington, DC in June 1994, was attended by ten legislators, legislative staff members 
and other key policy and decision makers. The group addressed several questions, including: what 
information is needed by legislators, at what points in time; what information gaps exist; and in what 
format is information most useful? 

The TA Center provided on-site technical assistance at a community service program, which, in addition 
to providing immediate help, was designed to elicit additional information about 
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the specific needs of customers, the most useful approaches to providing technical assistance, 
and the amount of time and follow-up required. 
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Results 

Family Members 

Information from family members shows that their knowledge of and involvement 
with evaluation activities varied markedly. Family involvement was typically related to satisfaction 
surveys or monitoring for federal block grant requirements. One major barrier to family involvement 
was lack of knowledge about evaluation in general, and absence of information about specific 
evaluations in their states or communities. 

Families hoped for evaluations in the areas of cultural responsiveness of services, service fit (in addition 
to consumer/family satisfaction), and tracking to assess whether those seeking services actually receive 
them. When they are involved at the design stage, families believe they can make important 
contributions clarifying the meaning of inquiries and the language used. Families reported that they need 
concrete examples of the role and importance of program and systems evaluation. 

Administrators of State Child Mental Health Programs 

All states were experiencing one or more major systems change, generally in multiple domains 
simultaneously, "nie developmental stage of changes varied markedly within and among states. 
Evaluation of systems change had been limited in all but a few states in the sample. The most common 
evaluation activities had been surveys of consumer/family satisfaction and level of involvement. 
Administrators reported that they desire evaluation to understand the progress, impact and effectiveness 
of changes as they are implemented in their states. 

However, while public children's mental health agencies have the responsibility to conduct evaluations, 
they need more technical expertise and resources to carry this out in a meaningful way. Although many 
states had in-house evaluation capacity, university affiliations, or both, such resources were often 
adult-oriented and of limited use for children's mental health systems evaluations. Primary areas of need 
for technical assistance for this group of stakeholders are illustrated in Table 1 . 



Legislators 

A number of key issues were identified by legislators participating in the focus group. They believed 
that too little is known about program outcomes; the crucial question "How does this service effect 
someone's life?" is answered too rarely. Legislators were also aware that basic data are often not 
available or not accurate. They acknowledged that even data about who is being served in which 
programs frequently are inconsistent or incomplete. 

Legislators wanted to know the individual successes or unhappy stories associated with services. They 
suggested that this information helps them understand programs better and helps them interpret 
information to their colleagues. They also wanted to know if a service is cost effective. A multiple 
system perspective is important to legislators because they must make decisions related to a number of 
child-serving agencies all of which have an impact on the others. They indicated that information about 
how changes in one systems effects other systems is helpful. 

Legislators reported a need for information at critical points in the legislative decision-making cycle. 
They need information that is concise and that is customized for the issue or discussion at hand. 

Community Site 

Our understanding of technical assistance needs was furthered through a pilot intervention with Project 
Connect, a local program in the Metro South Region of Massachusetts. This program integrates the 
efforts of agencies and families to develop and implement treatment plans for children with emotional 
disturbances. Through providing consultation on Project Connect's evaluation, as well as reviewing 
documents and interviewing staff, we learned more about the technical assistance needs of an evaluation 
in process. 
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1 . Time pressures. Needing data quickly for program improvement and funding agency 
requirements, Project Connect rapidly conducted an initial evaluation using focus groups and 
satisfaction surveys with parents, service providers, and referring professionals. We learned that 
longer-term planning for evaluation must be built around the need for quick turnaround on data. 

2. Limited resources for evaluation. Project Connect's limited resources made even modest 
evaluation difficult to implement. In addition to TA Center help. Project Connect recruited 
community volunteers to administer the surveys and graduate students to analyze data. We learned 
that agencies may need assistance to develop resources for basic evaluation fimctions. 

3. 

Small increments. Project Connect's modest efforts demonstrated the value of developing 
evaluation in small increments. The scope was feasible and widely acceptable, and the results 
were still usefiil. Family feedback, for example, which suggested overall satisfaction with the 
program, led to program improvements (e.g., more involvement of extended family members), 
and stimulated thinking about what child and family outcomes were reasonable to expect. This 
success encouraged additional evaluation efforts. We learned to support small improvernents in 
evaluation that take into account the time pressures and resource limitations faced by children's 
mental health systems and program. 
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Implications 

All representatives of stakeholder groups reported that they want data and outcome information to 
inform policy decisions and program development. They suggested that the most usefiil data are timely, 
concise, and presented in user-friendly language. They agreed that evaluation findings should be 
disseminated among policy and decision makers, service providers, consumers and families as part of a 
continuous feedback loop. Evaluation information can then be used to improve or change programs, 
make fimding decisions, and advocate for new services. 

Stakeholders also share a concern about selecting and measuring outcomes. States need assistance to 
determine which outcomes are appropriate for a state/community, select outcome measures, and gather 
the necessary data. Results of focus groups indicated that outcome information is desired across a 
number of domains, including child and family functioning, consumer/family satisfaction, service fit, 
cost effectiveness, and cross-system collaboration. Many stakeholders were concerned that existing 
instruments are too narrow and too deficit-based. 



There was agreement that children's mental health data are problematic. Data that do exist are embedded 
in the records of categorical agencies and do not give a holistic picture of the child and family. 
Computerized data systems need to be strengthened, both within mental health agencies and across 
agencies. 

Family-professional collaboration in evaluation has not been explored fully. We feel that, as key 
stakeholders, families must be brought more fully into the evaluation process at all points. Family 
members could benefit from information and training about how evaluation can be useful to them and 
how to become meaningful partners with other stakeholders around evaluation. Another issue that is 
central to the philosophy of systems of care is cultural competence, yet assessment of cultural 
competence in children's mental health is at a nascent developmental stage. The TA Center promotes 
evaluation as a collaborative effort, encouraging key stakeholder groups to participate in a range of 
evaluation activities. We believe that evaluations that are frilly participatory will reflect and respect the 
participants' cultural, social, and ethnic backgrounds. 

Findings from the needs assessment demon-strate the importance of providing a range of technical 
assistance activities that both demystify and promote evaluation to all stakeholders in children's mental 
health. The TA Center is fostering the use of evaluation to produce useful and timely information that 
will constructively inform and improve systems change. In many places the most useful evaluations may 
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be those that will provide information about collaboration and other structural changes, about 
consumer/family needs and satisfaction, and about intermediate, not long-term, child and family 
outcomes. Many states will need technical assistance to help them conduct evaluation in small, 
incremental steps, while developing a more comprehensive, long-term plan. 
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Introduction 

On July 1, 1993, Virginia implemented a major reorganization of services for youths with serious 
emotional and behavioral problems. Through the Comprehensive Services Act (CSA) for At-Risk Youth 
and Families, a system of services was to be created that was "child centered, family focused and 
community based." The CSA requires local interagency teams to plan and manage services and 
consolidated eight categorical funding streams to 
fund services. The new system of services is intended to: 

• increase interagency collaboration and family participation in service delivery and management; 

• dentify and intervene early with young children at risk and their families; 

• encourage public-private partnerships; 

• provide communities flexibility in the use of funds; and 

• reduce the use of more restrictive services, including out-of-home placements. 



A five-year evaluation plan was developed to be implemented in two stages. During the first stage, 
beginning July 1, 1993, the focus was on assessing the implementation of the CSA. During this stage of 
the study, the questions related to program implementation and interagency collaboration and funding 
were emphasized. Beginning June 1995, the research components related to assessing the impact of the 
projects on individual youths and families will be implemented. 

The focus of this report is on the implementation of the CSA in fiscal year 1994. 

Study Goal and Evaluation Questions 

The goal of this study was to examine those evaluation issues which were related to the implementation 
of the CSA. These issues included: 

Youths Served &shyp;The number and characteristics of the youths being served through the family 
assessment and planning teams (FAPTs) and/or receiving pool-funded services. 

Team Functioning and Interagency Co//a6ora//o«&shyp;The composition, coordination and 
collaboration of community planning and management teams (CPMTs) and family assessment and 
planning teams (FAPTs). 



,er|c 



32 



12/10/98 12:01 PM 



8th.23.html 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.23.html 



Family Participation&shyp',The involvement of family representatives and family members in the CS A 
process. 

Public-Private Partnership&shyp;What is the experience during the first year of closer working 
relationships between public and private agency staff? 

Goal ^c/»evemen?&shyp;CSA participants' opinions on progress made toward reaching the goals of 
the CSA. 
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Results and Implications 

Youths Served 

About 8,000 young people in Virginia were served through CSA pool funds during the first year. A 
majority (approximately 60%) of CSA youths were White, male and between the ages 
of 13 and 17. Most (91%) of the youths were "mandated," meaning they were entitled to services 
through federal and/or state mandates. 

Of all CSA youth served the first year who were mandated, 57% were in private, residential care. About 
66% of all youth served through CSA pooled funds were placed outside their homes. These figures 
include all foster care children. About 83% of all youth received services provided by the private sector. 
This figure, too, includes all foster care children. Six percent of mandated youth received 
non-residential, public services. 

Team Functioning/Interagency Collaboration 

A statewide survey mailed in April 1994 to members of all CPMTs and a sample of FAPTs addressed 
issues related to team processes and development. About 1,100 surveys were distributed statewide. 

There were 450 CPMT respondents and 89 FAPT respondents to the survey. 

Information gathered from CPMT chairpersons revealed that there had been positive collaborative 
relationships between their teams and the local school system, local government, and local judges, 
though there is room for improvement in the latter case. 

In addition, FAPT members were more likely than CPMT representatives to report that their team 
functioned cohesively, with a common purpose. Moreover, FAPT respondents agreed at a lower rate 
than their CPMT counterparts that they had a sufficient number of CSA training opportunities, and that 
they had adequate opportunities to learn about other agencies represented on their teams. 

A special study was designed to explore interagency collaboration among the five agencies participating 
on the CPMTs. Findings of this research suggest that local agencies had already set the stage for 
effective collaboration prior to the inauguration of a "comprehensive" approach to services. One area 
that seemed to need attention&shyp;and apparently received it during the first year&shyp;was the level 
of agreement among agencies regarding the needs of at-risk youth and families, and the services 
necessary to meet these needs. 

Two key factors in the study were predictors of success in collaborative efforts. One was 
"respon-sibility"&shyp;the degree to which agency represent-atives believed that each collaborating 
agency carried out its responsibilities and commitments to other agencies. The other factor was 
"satisfection"&shyp;the overall level of satisfaction members had with the other public agencies 
represented on their CPMTs. 

All five agencies received high ratings on responsibility from at least 60% of their peers. Likewise, all 
agencies received high ratings on satisfaction from at least 56% of their peers. These findings and others 
point to an enhancement of interagency collaboration through the CSA team processes. Moreover, it is 
likely that agencies will engage in future collaborative efforts to address the needs of troubled youth and 
their families in Virginia. 
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Family Participation 

FAPT and CPMT members were surveyed about parent representatives on their teams. Responses were 
generally positive. 

• 87% of CPMT respondents and 82% of FAPT respondents agreed that meetings were scheduled at 
times parents could attend. 

• 79% of CPMT respondents and 88% of FAPT respondents agreed that parent representatives 
participated in team decision-making. 

• 74% of CPMT respondents and 78% of FAPT respondents agreed that parent representatives had a 
good understanding of the problems of at-risk children and youth and their families. 

• 65% of FAPT respondents agreed that participation of the parent representatives on FAPTs had 
resulted in improved service plans for at-risk youth and their families. 

• 59% agreed that services have improved with parent participation. 

• FAPT members were asked a set of questions about participation in the CSA process by parents of 
the youth served. Their responses were also generally positive. 

• 79% reported that parents attended FAPT meetings at which their children were discussed. 

• 82% said that the parents actively participated in these meetings. 

• 94% agreed that the presence of family members at FAPT meetings helped to ensure that the 
views of the youth's family were adequately represented. 

Public-Private Partnership 

CPMT survey respondents provided the following information about their perceptions of privately 
provided services. 

• 65% of CPMT respondents reported that privately provided services were available for local 
clients. 

• 64% said the availability of private services had not increased since the implementation of the 
CSA on July 1, 1993. 

• 70% thought that privately provided services were of "high quality." 

• 75% agreed that private agencies "demonstrated considerable knowledge" of the service needs of 
at-risk youth and their families. 

Finally, CPMT chairpersons were asked their perceptions of fees charged by private providers. Of the 53 
chairpersons who indicated having knowledge of private provider rates, 28 (53%) perceived an increase 
in fees since the beginning 
of the CSA. 

A majority of all survey respondents saw no increase in private services during the first year of the CSA. 
Such a change in the availability of services could not be expected to happen quickly, nor would it 
happen automatically, especially in areas which had no history in, nor infrastructure for providing 
adequate services to children. Nevertheless, the founders of the CSA hoped that private providers, 
working closely with public agency staff and parent representatives, would increase their services, 
especially those that were community-based. 

Goal Achievement ^ 
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Among the seven major CSA goals, the one which received the most recognition of progress 
by bo A CPMT and FAPT survey respondents 

was interagency collaboration. There was also considerable acknowledgment of progress made in 
incorporating families into the service plarming processes of the CSA. 

A majority of respondents perceived at least some or moderate progress toward the achievement of three 
goals; 



• provide communities flexibility in the use of funds; 

• provide services in the least restrictive environment; 

• improve the quality of services to troubled youths and their families. 

The two goals for which all respondents reported the least amount of progress achieved were 
public-private partnerships and early identification of, and intervention with, at-risk young children. It is 
understandable that more than 9-12 months would be needed to make substantial progress in these areas. 
In fact, it was surprising to observe the overall amount of goal achievement perceived by CSA 
participants during its first year of operation. 
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Conclusion 

In summary, survey respondents feel that the CSA has significantly changed the way in which Virginia 
addresses the emotional and behavioral problems of its young people. This report examined the 
successes and shortcomings of the CS As first year of implementation. The focus 
was on the perceptions of those most directly responsible for the day-to-day operations of the CSA. The 
information they provided indicated that a tremendous amount of individual and group effort was 
expended to inaugurate the CSA. These efforts, moreover, led to significant accomplishments in a short 
period of time. Most notably, a high level of collaboration among agencies existed throughout most of 
the state. Through the CSA process, agency staff coordinated their efforts 

to deal with at-risk youths and their families. Moreover, survey respondents indicated that parents, both 
as community representatives and as guardians of troubled youth, have been meaningfully included in 
the process. 
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Introduction 

Access Vermont is part of the Comprehensive Community Mental Health Services Program for 
Children, a national services initiative funded through the Center for Mental Health Services (CMHS) 
under the Family Preservation Act to provide children's non-categorical mental health services in 22 
sites across the country. The grant will last for 5 years and began in October, 1993. Vermont is one of 
two states to receive statewide funding through tWs initiative. This paper will provide an overview of 
Access Vermont implementation and the evaluation component of Access Vermont with particular 
emphasis on: 

1 . the creation and implementation of an evaluation process that is individualized and which is 
beneficial to each local community; 

2. helping communities to incorporate evaluation into service provision; 

3. helping communities to empower children and families; and 

4. approaches to measuring outcomes of community-based, non-categorical services. 



Access Vermont: Implementation 

Background of Non-categorical Services in Vermont 

Over the past decade, Vermont has made considerable progress in developing a statewide, community 
based system of care for children and adolescents experiencing severe emotional and behavioral 
disturbances and their families. On the individual services level, Vermont developed a statewide 
capacity to provide community-based wraparound services with funding from Robert Wood Johnson 
(RWJ). On the systems level, funding from CASSP allowed each of the twelve local areas in Vermont to 
establish Local Interagency Teams (LITs) made up of parents, community members, and local agency 
representatives. 

Access Vermont has built upon these systems of care established under RWJ to help provide 
community-based, locally planned and implemented, non-categorical services. Through these systems of 
care, Vermont has been able to reduce the rates of out-of-school-district, out-of-home and out-of-state 
placements of children and youth, as well as the numbers of children taken into state custody (V ermont 
Department of Mental Health and Mental Retardation, 1993). 

Through the implementation of innovative programs and interagency teams, Vermont's mental health 
service system has been effective in serving many of the state's most challenging children and youth in 
their homes and communities, rather than in more restrictive treatment settings. 
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Long Term Goals of Access Vermont 

The long term goals of Access Vermont are to continue to reduce the rates at which Vermont's children 
are being removed from homes, schools and communities to more restrictive treatment settings, and to 
demonstrate better outcomes and reduce costs in the process. These goals will be accomplished through: 

1 . increasing the quality of services available for children and families in home and school settings, 
with particular emphasis on effective and timely crisis response; 

2. evaluating and feeding back consumers' responses about their satisfaction with services and their 
empowerment within the system; and 

3. effectively building evaluation, with an eye towards outcomes, into service provision. 
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Access Vermont: Evaluation 



Development 

While the Local Interagency Teams have served primarily as case review teams in the past, the LITs are 
now moving towards policy making, by taking primary responsibility for local interagency plaiming, 
coordination and services integration. Through the LITs, community based services have been designed 
by local areas to meet their individual needs and to build on existing strengths. At this writing, 1 1 of the 
12 proposals for services have been reviewed and approved by the Vermont Department of Mental 
Health. Each catchment area receives money to implement these Local Priority service plans. These 
individualized services, inspired by community needs, are the heart of Access Vermont. 

Services that are being implemented under Access Vermont by local areas include crisis outreach and 
stabilization, crisis hotlines, short-term residential crisis stabilization programs, family crisis response 
teams, respite care, day treatment programs (school-based programs, after school and summer programs, 
day programs for transition aged youth), detoxification facilities, transition services, and a family 
support center (see Figure IT 

Evaluation 

With all the different, individualized services being implemented and offered under Access Vermont, 
evaluation can be a challenge. Therefore, evaluation is being conducted on two levels&shyp;the 
individual level of the child and family, and the systems level of the service providers. 



Consumer Evaluation 

A vital part of Access Vermont is helping communities to empower children and families, and helping 
communities to evaluate empowerment and satisfaction. Families wall be asked about their 
empowerment and satisfaction wdth prior and current services through measures (Family Empowerment 
Scale, Koren, DeChillo & Friesen, 1992; Client and Youth Satisfaction Questionnaires) administered 2 
weeks after the start of multi-agency services by members of the Evaluation Team (staff from the 
Vermont Department of Mental Health, the University of Vermont, and a Parent-Interviewer, hired 
through the Vermont Federation for Families). 



Demographic information and a number of standardized measures (CBCL: Achenbach, 1991 ; Youth 
Self Report: Achenbach, 1991; Child and Adolescent Functional Assessment Scale: Hodges, 1994; and 
Residential Living Environments Scale: Hawkins, Almeida, Fabry & Reitz, 1992) will be completed at 
intake by service providers for all consumers receiving services under Access Vermont. Follow-up 
interviews, consisting of the empowerment, satisfaction, and standardized measures will be conducted 
by the Evaluation Team wdth a random sample of consumers at 6 months, 1, 2, and 3 years after Intake 
for those consumers still receiving services. Youth and parents wall be reimbursed individually for their 
efforts in these follow-up contacts. 
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Addressing the Evaluation Needs of Local Areas 

All of the measures listed above vvill be part of the evaluation at all 22 sites. However, the evaluation 
team is also working with each of the 12 local sites to evaluate any additional outcomes that the site may 
want to address. 

For example, one predominantly rural area that is implementing crisis outreach services wants to reach 
previously imserved, rural populations, and to assess how many "saves" they accomplish&shyp;that is, 
how many children and youth they successfully prevent from crises that would have result in 
out-of-home placements. Their evaluation centers aroimd these issues, as well as evaluation of prior 
service involvement to assess how many previously imserved consumers they are reaching. 

Some areas will also be working closely with the schools to evaluate educational outcomes. The 
evaluation team is working with these areas to help them get the schools involved, and use outcome 
measures that will best serve their needs. 



System Evaluation 

Each LIT will serve as the hub connecting the local network of services used by children experiencing 
severe emotional disturbance. In order to evaluate how the different interagency systems are working 
together, LIT members will complete the Local Interagency Collaboration Checklist (Froelich, 1992) at 
the implementation of Access Vermont services, and annually thereafter for 3 years. Information 
obtained from this measure will be fed back to local areas. 

Feedback 

As both direct service providers and researchers know, service provision and evaluation have often been 
viewed as mutually exclusive and therefore conflicting interests. Evaluation outcomes are often not seen 
by service providers, and all too often, research findings are not applied to create positive organizational 
or service delivery changes. In the Access Vermont project, we are committed to a collaborative model 
where research, evaluation, and outcomes will blend ivith the innovative services that are being provided 
by local areas under this project, to maximize the benefits for both service providers and consumers. 

One of the most important steps in this collaborative process is the creation of a feedback loop, where 
service providers will receive timely periodic feedback on service outcomes and empowerment 
measures. Feedback will include not only standardized measures, but specific outcomes that individual 
communities have identified as important. 

We feel that one of the most effective ways to blend services and evaluation is to incorporate 
standardized assessment and outcome measures into intake procedures and clinical planning. By 
working collaboratively with service providers, assessment tools such as the CBCL can become 
beneficial in assessment procedures and ongoing clinical planning. A vital component of both local 
ownership and clinical relevance is helping service providers to utilize the standardized measures for 
assessment and treatment planning. The evaluation team will work to provide local areas with training in 
the clinical use of the assessment measures. We believe that empirical assessment measures will not only 
help to evaluate effective services, but can also aid in the provision of the most effective and relevant 
services. 

Quarterly data repoits will also be sent to local areas to give summary feedback on all measures for the 
consumers they have served. These reports will help service providers gain a broader picture of the 
children and families they are serving, the types and hours of services Aey are delivering, and the 
difficulties and challenges of the consumers they are serving. Data will also be fed back in customized 
reports, in ways that local areas and agencies have deemed most useful to them. 

One exciting project the evaluation team has begun work on is the Access Vermont Newsletter, filled 
with items that are relevant to the project. We are encouraging LIT members, service providers, and 
anyone else involved with Access Vermont to contribute ideas or items to the newsletter. We are very 
excited about this project and hope that it will aid in feedback and communication between local areas 
and the evaluation team. 
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Conclusions 

Evaluation of non-categorical mental health services for children and families as irmovative and as 
diverse as those planned under Access Vermont is certainly a challenging task. The evaluation itself has 
several aspects; it must support non-categorical children's mental health services, empower families and 
commimities, and provide meaningful outcome data. In keeping with the system of care philosophy of 
family and commimity empowerment, both the services and the evaluation of those services are being 
designed by a collaboration among families, commimities, mental health systems, and experienced 
researchers. 

It our belief that sharing our experiences in the design and implementation of this evaluation will be 
valuable, not only to the service systems and consumers of Vermont who are a part of Access, but to 
other communities, service providers and researchers who are implementing and/or evaluating similar 
services. 
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Introduction 

The Southern Consortium for Children (SCC; a regional mental health authority) and local county 
administrations have gradually changed the mental health service delivery system in Southeastern Ohio 
from a state-operated hospital model to a community-based, community-directed model. Program 
development has centered around the creation of children's mental health case management, therapeutic 
foster care, home-based services, and other services critical to providing a regional system of care. 
However, little information regarding the impact or effectiveness of system changes is available. With 
the emphasis on evaluation (Stroul & Friedman, 1994; both in terms of rigorous methodologies and 
naturalistic studies of program effectiveness) along with recent health care reform and the push for 
consumer driven service delivery (Burchard & Shaefer, 1992) the SCC increasingly interested in 
developing methods of ongoing evaluation. 

Conceptualization of Outcome 

This paper represents the initial efforts of the SCC to develop a system for ongoing evaluation of clinical 
outcomes for youth receiving services in collaboration with Ohio University. To address region specific 
needs, four areas of outcome assessment were considered: (1) problems associated with rural service 
provision; (2) perspectives of various stakeholders; (3) current research methods of outcome 
measurement; and (4) a theoretical and conceptual scheme of outcome assessment. 

Unique Problems in a Rural Setting 

Southeastern Ohio presents several unique problems for the provision of services and the development 
of an evaluation program. The service area is composed of ten counties located in the foothills of the 
Appalachian Mountains and in the heart of Ohio's 29 designated Appalachian counties. The terrain is 
rough and marked with poor roads and a non-existent public transportation system. The region is 
characterized by a sparse population, economic underdevelopment, and a low tax base. Not only do 
these difficulties impact the provision of services, but they necessarily affect the implementation of any 
outcome evaluation program. Evaluating service effectiveness must rely on brief, inexpensive methods 
of assessment that have intuitive value for the consumer. 

The Input of Stakeholders 

Strupp and Hadley (1977) proposed a tripartite model of mental health outcomes in which three 
interested parties consider the outcome of mental health interventions: society, the client (or consumers), 
and the mental health professional. Based on the viewpoint of the stakeholder (Gold, 1983), different 
criteria are selected to measure successful treatment. An initial step in establishing an evaluation plan is 
to make inquiries into these populations with regard to what kind of outcomes they deem are important. 
We used a Social Validation Questionnaire (VanDenBerg, Beck, & Pierce, 1992) to provide input from a 
variety of viewpoints regarding outcome assessment in 
our region. 
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Research Input 

Studies investigating the effectiveness of mental health services for children were also examined to aid 
the outcome development plan (e.g., Duchnowski, Johnson, Hall, Kutash, & Friedman, 1993). The 
review focused on instruments used to evaluate outcome and identified relevant domains of outcome. 
While all areas of assessment were not included, reviewing well-designed studies helped to ascertain the 
most important domains of assessment. 

Use of a Theoretical and Conceptual Measurement Classification Scheme 

Because of the divergent processes that occur during mental health intervention, divergent methods of 
measurement are used to capture the complexity of human functioning and change. Selecting from 
among these methods may be difficult if not guided by a theory. An ideal scheme may give purpose and 
direction to the selection of a final assessment package. The conceptual scheme used in this plan 
(Lambert, Ogles, & Masters, 1992) includes four theoretical dimensions upon which outcome 
instruments vary: (1) the content area addressed by the instrument, (2) the source of outcome ratings 
sampled by the instrument, (3) the outcome instrument's method or technology of data collection, (4) 
and the time orientation or stability of the instrument. The scheme was used to help select and develop 
an outcome assessment package that would tap a variety of content, source, technology, and stability 
dimensions. 
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Summary of Conceptualization 

Based on theses considerations, a list of desirable characteristics for measuring clinical outcomes was 
developed: 

1 . Outcome assessment must be pragmatic in terms of time and expense. Many research-based 
instruments are too cumbersome for ongoing clinical use. 

2. Outcome data must be understandable for paraprofessionals, parents and children. 

3. Effective outcome assessment should include input from multiple sources. 

4. Multiple content areas of outcome should be considered. 

5. Pragmatics should be counterbalanced by psychometric properties, including: reliability, validity, 
and sensitivity to change. 
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Development and Initial Evaluation of the Ohio Scales 

Based on our conceptualization, a set of brief, pragmatic outcome measures were developed. Three 
parallel forms of the Ohio Scales (OS) were developed for completion by the youth's parent or caretaker 
(OS-P), the youth (OS-Y), and the agency worker (OS-A). Four domains of outcome were selected: 
severity of problems, level of functioning, hopefulness, and satisfaction with treatment. Parents and 
youth rate all four areas. Case workers rate the youth using the problems and functioning scales. 

To begin evaluating the psychometric properties of the instrument, two samples of data were collected. 
For the community sample, approximately 297 Jr. High and High School students (average age 14.36, 
SD 1.54) completed the OS-Y. In addition, 281 parents (average age 39.43, SD 7.36) of each respective 
youth completed the OS-P. For the clinical sample, case managers at two agencies rated a total of 78 
youth (average age 14.28, SD 2.16). Additionally, 16 youths from this population completed the OS-Y 
and Youth Self Report (YSR; Achenbach & Edelbrock, 1983) and 28 parents completed the OS-P and 
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Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1983). Table 1 provides basic descriptive 
statistics for the OS problems, hope, and functioning domains for each respective population. 

Reliability 

Internal consistency data for each OS category is presented in Table 2. The internal consistencies for 
each category are adequate. Additional projects are underway to assess the inter-rater and test-retest 
reliability of the Ohio Scales. 

Validity 

Evidence of the concurrent, discriminant, and construct validity of the OS was also collected. Parent and 
youth ratings of problem severity and functioning were correlated with the CBCL and YSR (see Table 
3). The CBCL and YSR were included primarily to assess concurrent validity of problems. Additional 
studies are underway to validate the functioning scales. Case manager ratings of functioning and 
problems were correlated with the Progress Evaluation Scales (PES; Ihilevich & Glesser, 1979). 

The community sample provides some evidence for the discriminant validity of the OS. Five t-tests were 
conducted using parent ratings of functioning as the dependent variable to examine differences between 
students who had repeated a grade, been arrested, received mental health services, been assigned to 
classes for students with behavioral problems, or assigned to classes for students with learning problems, 
and those who had not experienced these events. Students who had been assigned to classes for youth 
with learning difficulties, had received previous mental health services, or had been arrested had 
significantly poorer functioning and more severe problems than students who had not experienced these 
events. Students who had previously been assigned to classes for youth with behavior problems had 
poorer functioning (but not more severe problems) than students who had not been assigned to these 
classes. 

The PES were administered in conjunction with the OS-A in the clinical sample on two 
separate occasions (approximately 4 months apart) to assess sensitivity to change. Change scores on the 
OS-A problems and functioning scales significantly correlated with the PES (r = .54, and .56, P < .001) 
and indicate a relatively high degree of agreement between the two instruments with regard to client 
change. 
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Conclusion 

While by no means completed, the results of the initial investigation of the psychometric properties of 
the OS are quite positive. The Ohio Scales have adequate internal consistency and preliminary evidence 
of concurrent and construct validity along with sensitivity to change. Additional projects are underway 
to further examine these properties. By conforming to the rather stringent conceptual outcome 
requirements, the developers hope that the final result will be pragmatically useful yet methodologically 
rigorous outcome measures that can be used for practical, ongoing evaluation of services for children 
and youth. 
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Introduction 

The Child and Adolescent Service System Program (CASSP; Day &. Roberts, 1991; Stroul &. Friedman, 
1986), which received initial federal funding in 1984, was established to assist states and communities in 
their efforts to improve child and adolescent service systems. CASSP has promoted the development of 
systems of care that are child-centered and family focused, with the needs of the child and family 
dictating the types and mix of services provided. 

In 1988, when the California State Department of Mental Health (Cal-DMH) received federal funding to 
implement the system of care principles developed by CASSP, a system of care program in Ventura 
county had been successfully in place for several years and the positive effects on children's mental 
health services had been demonstrated (Feltman &. Essex, 1989; Jordan &. Hernandez, 1990). Also in 
1988, California State Assembly Bill 377 (AB 377) had been passed to expand the Ventura program into 
three additional counties (Attkisson & Rosenblatt, 1993; Rosenblatt & Attkisson, 1993; Rosenblatt & 
Attkisson, 1992; Rosenblatt, Attkisson, &. Fernandez, 1992; Ventura County Health Care Agency, 

1989). Therefore, the federal CASSP grant was seen by Cal-DMH as an opportunity to support and 
expand local system of care efforts while attempting to accomplish similar reforms at the state level. 

In 1993, an evaluation of the impact of CASSP on California's mental health service system for children 
and adolescents vvdth serious emotional disturbance was conducted. Objectives included (1) evaluation 
of CASSP efforts to develop interagency and interdepartmental coordination and collaboration, (2) 
evaluation of CASSP efforts to develop family participation and advocacy related to services for 
children and adolescents vvdth serious emotional disturbance (SED), and (3) evaluation of CASSP efforts 
to promulgate cultural competence in systems of care for children and adolescents. A comprehensive 
description of the procedures and results of this evaluation have been previously reported (Madsen, 
Dresser, Rosenblatt, &. Attkisson, 1994); the current analysis will focus on the similarities and 
differences between informant groups (county human service agency staff and parents of children and 
adolescents vvdth SED) in their reports of the impact of CASSP and the changes in California's service 
delivery system over the last four years. 
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Method 



To accomplish the evaluation objectives, three studies were conducted: (1) face-to-face interviews with 
county level agency staff, (2) telephone interviews vvdth members of a statewide parent organization, and 
(3) a comprehensive review of relevant federal, state, county, and local documents. Selected results of 
Ae first two studies vvdll be presented here, while results of the third study 
are presented in a companion paper (Dresser, Clausen, Rosenblatt, &. Attkisson, 1995). 
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In the first study, of the 86 persons identified by state and county mental health staff as potential key 
informants in the selected 21 counties, interviews were conducted with 60 respondents (70%). Subjects 
for the second study were members of a statewide organization of parents and family members of 
children and adolescents with SED, the Family Network of California (FNC). Interviews were 
conducted with 67 respondents (31.5% of the identified sample) across 23 counties in California. 

Measurement 

A 45-minute, semi-structured interview was developed for use with management and program staff in 
county departments of mental health, social services, juvenile justice, and education. Respondents were 
asked to estimate the impact of California CASSP in several areas, as well as to rate the changes in 
county service systems, using a Likert-type scale of 5 items, ranging from 1 (none or no impact) to 5 
(extensive impact). Informants were also asked to respond to several open-ended questions regarding 
changes in service systems for children and adolescents with SED over the last four years. 

The face-to-face semi-structured county interview format became a model for development of a 
semi-structured 30-minute telephone interview for use with parents of children and adolescents with 
SED. 
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Results and Implications 

Interagency Collaboration 

When asked to rate the impact CASSP had on increasing interagency collaboration in the delivery of 
services to children and adolescents with SED, both agency staff and parent informants indicated, on 
average, that CASSP had some impact in this area; however, the parent sample appeared divided on this 
issue, with many rating CASSP's impact as significant (25%) and many others rating it as minimal 
(17%). All of the county interviewees responded yes when asked if agencies in their county were 
working together in important ways; however, only 60% of the parents interviewed responded yes to this 
question. Overall, interview results suggest there is a perception that collaboration between agencies 
providing human services to youth with SED increased modestly throughout the State of California 
during the four years of CASSP implementation. 



Family Participation 

When asked to rate CASSP's impact on parent involvement in service development, implementation, and 
evaluation, county informants indicated a greater impact (mean = 3.36 on a scale of 1 - 5) than did parent 
informants (mean = 2.93 on a scale of 1 - 5), with most people in both groups (68% of county 
informants and 58% of parents) indicating CASSP had some (> 3 or some on a scale of 1-5) impact in 
this area. As compared to four years earlier, 82% of county agency informants and 58% of parent 
informants felt that service providers were more aware and supportive of parents and families of youth 
with SED. Thus, although subject responses to open-ended questions demonstrated that families of 
youth with serious emotional disturbance perceive human service agencies as inaccessible and 
frustratingly complex, there is evidence that CASSP had an impact on increasing the involvement of 
parents in human service planning and delivery. Analysis of comments made by county and parent 
interviewees suggests that CASSP's impact on increasing family participation was especially notable in 
the areas of staff and parent training, empowerment of individual parents, and organization of local and 
state parent groups. 



Cultural Competence 

Informants were asked to rate the extent to which their county valued cultural competence in service 
systems for children and adolescents with SED. It is in this area that county agency staff and parents of 
youth with SED most differed in their perception of county service systems. Indeed, almost two-thirds of 
county agency staff reported that their county placed significant or extensive value on moving systems 
of care toward becoming more culturally competent while two-thirds of parents reported that their 
county placed no, minimal, or some value on this issue. One possible interpretation of these findings is 
that, although service agency persormel have become increasingly aware of the importance of making 
service delivery culturally competent, changes have not yet been implemented at the direct service level 
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and have therefore not been perceived by service consumers and their families. Overall, results suggest 
that the perceptions of modest improvements in the cultural competence of service delivery were due in 
part to CASSP's efforts to inspire county mental health administrators to become leaders in the 
movement to establish culturally competent systems of care for children and adolescents. 
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System Change and Outcome 

\^en asked whether or not they believed systemic changes had occurred in their county's service 
delivery system for children and adolescents with SED, almost three-fourths of county agency staff and 
over half of parent informants responded yes. However, when asked to rate the extent to which the 
human service needs of children and adolescents with SED in their county were being addressed, over 
two-thirds of county agency staff informants and almost three-fourths of parent informants reported that 
those needs were addressed minimally or not at all. Several respondents noted that, although many of the 
needs of children and adolescents who have been identified are being addressed, there are many more 
youth with SED who are not yet identified by the system. 

In summary, the major findings regarding the perceptions of county service agency staff and parents of 
children and adolescents with SED about the impact of CASSP in California and changes in systems 
which serve youth with SED are as follows: 

1. a majority of both groups state that CASSP has had some impact on improving interagency 
collaboration between county human service agencies; 

2. both groups feel that human service agencies are working together in important ways, though 
human service agency staff are more in agreement with this statement; 

3. both groups feel that CASSP has had some impact on parent involvement in mental health service 
systems for children with SED; 

4. generally, both human service agency staff and parents of children with SED state that mental 
health service providers are more aware and supportive of parents and families of children with 
SED than they were four years ago; 

5. human service agency staff report the perception that agencies highly value cultural competence in 
service delivery while parents of children with SED report that agencies minimally or somewhat 
value this concept; 

6. both human service agency staff and parents of children with SED feel that there has been a 
systemic change in service delivery systems to children with SED over the last four years; and 

7. on average, human service agency staff feel that the needs of children with SED are being 
adequately met, while the majority of parents state that such needs are only minimally met. 
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Introduction 

The Child and Adolescent Service System Program (CASSP) in California is designed to compel system 
of care development for children and adolescents with serious emotional disturbance. As part of the 
evaluation of die CASSP in California (Madsen, Dresser, Rosenblatt & Attkisson, 1994), a review of 
documents was conducted to gain an understanding of the status of human service reform efforts in 
California. Documents reviewed include: (1) federal system of care policy and program descriptions; (2) 
state memoranda, documents, and reports relevant to CASSP implementation in California; (3) reports 
from county administrators involved with system of care development; and, (4) child advocacy and 
parent support group newsletters and monographs. The goal of this review of written products was to 
provide a broad context within which to assess 
CASSP contributions in California. 

Return to Top 



Method 

Documents were identified for review after consultation with state and county administrators, human 
service and research professionals, parents, and child advocates. Additional documents were also 
selected from the emerging literature on the design and evaluation of systems of care in California. 
Altogether, thirty-five documents representing over 7,500 pages of text were selected for review. The 
documents selected for review were organized into four broad types to enable reviewers to consider 
aspects of service system development from the perspectives of federal, state, county, and consumer 
participants in the reform process (See Table IT Reviewers developed detailed notes on the content of 
each document. From the notes, major themes, continuities, and discontinuites across documents were 
identified by project staff 

and synthesized into a summary of document contents. 
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Results 

The results section provides a summary of major themes and continuities regarding human service 
reform in California. While there was some variation in coverage and emphasis across documents (see 
Madsen, Dresser, Rosenblatt & Attkisson, 1994), the content of the documents relate to one of three 
themes: (1) challenges facing California; (2) factors that facilitate human service reform in California; 
and, (3) CASSP contributions in California. 

Challenges Facing California 

California faces many challenges to state-wide human service reform. Described below are a few of the 

O 

,ERIC 



53 



12/10/98 12:08 PM 



8th.45 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.45.html 



challenges discussed by document authors. 

1. Geographic and Population Scope and Diversity. California's population is nearly 32 million people. 
The state has the seventh largest economy in the world G^)*. There is a diversity of geopolitical regions 
(urban, suburban, rural, mixed) and extraordinary ethnic diversity in many parts of the state. Population 
growth is driven in great part by immigration. Most immigrants to California are ethnically diverse 
people of child bearing age from Mexico, Central and South America, Pacific Rim, and Asian lands (4). 

Geographically large, densely populated, and ethnically diverse regions of the state with large numbers 
of low income residents (e.g., Alameda, Los Angeles, and the large southern counties of the state) pose 
special challenges to adequate service provision. Human service providers are challenged to become 
more knowledgeable and sensitive to the culturally specific needs of ethnically and economically diverse 
populations of children and families. Public schools in a number of California counties face complex 
problems stemming from large numbers of non-English language speaking children coupled with 
inadequate numbers of cross-culturally trained and hi- or multi-lingual educators (9). 

2. Economic Recession and Erosion of the Tax Base. The last national economic recession produced 
severe contractions in California's construction, computer, and defense industries and resulted in a 
decline in the average earnings of Californians. Hardest hit were the lowest 40 percent of wage earners, 
the unemployed, and the very poor who are most dependent on public services (3). 

The erosion of California's tax payer base provides less opportunity for local governments to keep pace 
with increased case load demands for public service. Voter and legislative initiatives, such as the 
Proposition 1 3 limitations on property taxation, the Proposition 98 set-aside of 40% of state funds for 
K-12 education, and the Garm limits on state spending, have severely curtailed the proportion of the state 
budget available for discretionary spending. Increased demand for public services is difficult to 
accommodate in California, where over 90% of the state budget is allocated to fixed costs and 
entitlement programs. 

2. Complex, Convoluted and Inflexible Funding. Health and human service administrators in California 
must navigate a complex array of federal, state, and local funding streams and private sector (insurance 
or foundation) support to marshal resources on behalf of children and youth. There are no less than eight 
federal funding streams relevant to mental health services alone, each of which finance very specific 
services to a very narrow target population of youth. 

A study conducted by an external consultant to the California State Department of Mental Health (2) 
revealed that California could significantly expand resources available for prevention efforts and 
community based services by lifting self-imposed limitations on access to and use of federal entitlement 
fimding streams. Additionally, financial incentives are typically structured into the formulas for 
calculating in-home vs. out-of-home placement rates for each public human service sector in California 
&shyp; mental health, child welfare, juvenile justice and special education. Across 
the board, state and county cost sharing ratios produced incentives for counties to shift costs to the state 
through increased reliance on out-of-home placements (11). 

4. Lack Of Objectives and Accountability. Human service delivery systems throughout 
the state generally suffer from a lack of well-defined objectives, and a lack of accountability for 
documenting child, youth, and family outcomes. Without clear service objectives, and criteria for 
evaluating effectiveness, administrators and providers operate within a performance and information 
vacuum. 

Factors that Facilitate Human Service Reform in California Factors that facilitate human service reform 
in California include policy, fiscal, and programmatic changes that have been influenced by national 
trends, funding opportunities, state legislation, activist government, and parent leadership. Below are a 
few of these factors. 

1. Mental Health Program Realignment. Program Realignment (AB 948, the Bronzon-McCorquodale 
Act of 1991) is a legislative change in California with far reaching consequences for the financing. 
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organization and delivery of mental health services. Program Realignment was prompted by the state 
budget crisis (a $14 billion dollar state budget deficit in FY 1991-92) and constraints on the state budget 
process limiting discretionary expenditures for health and human services. The bill realigns fiscal 
responsibility by shifting costs, control, some revenue generation, and responsibility from the state to 
counties for public mental health, public health, some social services, and some state psychiatric hospital 
and locked facility beds. The cost shifting involves over $2 billion dollars worth of service provision 
annually (3). Realigned programs are funded through increased vehicle license fees and a permanent 
one-half cent sales tax increase. Formal research and evaluation of Program Realignment in California is 
not yet complete. 

However, an interim assessment (7) notes improvements in state financing of mental health services 
derived from Program Realignment. The improvements stem from: (a) a guaranteed revenue stream 
which enables county administrators to plan ahead and county governments to serve as more reliable 
contracting partners; (b) relaxed financing restrictions of state bureaucracy which enables counties to use 
de-categorized Medi-Cal and private funds more aggressively and carry over unexpended funds from 
one fiscal year to another; and, (c) increased county fiscal responsibility for service provision which 
shifts the incentive structure away from high rates of utilization of expensive institutional services. 

2. Managed Mental Health Care. The California Department of Health Services (DHS) managed care 
initiative (6) integrates California's two Medi-Cal funding streams for mental health services 
(Short-Doyle/Medi-Cal and Fee-For-Service/Medi-Cal) and places Medi-Cal mental health services 
under the domain of county mental health departments. Counties have a choice to serve as the Local 
Managed Mental Health Care Plan (LMMHCP) offering an array of services that emphasize prevention, 
promotion of health and functional ability, and decreased reliance on hospital and institutional services. 
In the managed care plans, individual providers are linked together under the umbrella of the LMMHCP. 
All eligible Medi-Cal beneficiaries receive their mental healA services through the LMMHCP in 
accordance with access criteria and standards set by the state and Medicaid. 

3. System of Care Development. Legislatively enabled, state funded, and independently evaluated 
systems of mental health care are evolving in an expanding number of California counties (Rosenblatt, 
Attkisson & Fernandez, 1992; Rosenblatt & Attkisson, 1992, 1993; Attkisson & Rosenblatt, 1993; 
Attkisson, Dresser, 8c Rosenblatt, in press). System of care efforts were inspired 

and given definition by leaders from Ventura County in conjunction with local and state political 
representatives. Building upon CASSP principles, the Ventura leaders devised a planning model for 
reform of mental health service delivery (Feltman & Essex, 1989; Jordan & Hernandez, 1990). 

Following the planning model, Ventura successfully implemented community based mental health 
services for disturbed youth, demonstrated improved youth outcomes, and decreased overall 
expenditures. The initial 1984 demonstration in Ventura County (AB 3920) was expanded in 1987 to 
Riverside, San Mateo, and Santa Cruz (AB 377). More recently, through new legislation (AB 3015 
passed in 1993), the California Model System of Care was extended to an additional seven counties: 
Humboldt, Los Angeles, Merced, Monterey, Placer, San Luis Obisbo, and Stanislaus. San Francisco 
county pursued a system of care demonstration in the Family Mosaic Project with grant funds from the 
Robert Wood Johnson Mental Health Services Program for Youth (8). 

Collectively, system of care counties work toward desired outcomes for targeted populations of youth 
with serious emotional disturbance: controlled rates of out-of-home and state hospital placements and 
associated lower costs, improved school attendance and academic performance, 
and decreased juvenile justice recidivism. The system of care counties are implementing community 
based mental health services that involve improved financing for consumer choice and flexibility and 
individualization of service delivery. Parental, extended family, and foster family participation in 
program and treatment planning is an essential component of the California system of care movement. 
Other programmatic features include centralized and strategic case management to avoid unnecessary 
out-of-home placement, multi-agency coordination and treatment team approaches, and cultural 
competence of service delivery (10). The combination of demonstrating positive client outcomes while 
controlling public costs for services provided through multi-agency collaboration has facilitated 
tremendous support and momentum for systems of care in California. 

CASSP Contributions in California 
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Significant CASSP contributions in California include: 

1. Dissemination of System of Care Principles. Information dissemination regarding system of care 
concepts, along with funding opportunities for local system of care development, produced the 
knowledge and incentive systems necessary for the formation of a critical mass of system of care 
proponents throughout California. The CASSP provided organizing energy, technical assistance and 
formal trainings to coimty administrators, human service professionals, and parent groups seeking to 
improve local systems of care, enhance cultural competence of service delivery, and increase parent 
involvement in service systems for children and youth with serious emotional, behavioral and mental 
problems. Since CASSP development in California, system of care principles became an organizing 
theme for mental health service system development as reflected in the California State Mental Health 
Plans, the State Master Plan, coimty planning documents, system of care expansion and evaluation, and 
shifts in Medicaid ^ding policy. The system of care philosophy has been disseminated successfully 
among the California Mental Health Planning Council and the California Mental Health Directors 
Association, along with other key state-wide policy and planning groups. 

2. Formation of Broad-Based Advisory and Advocacy Groups. The CASSP supported the development 
of a state-wide multi-agency policy process through the role of the State 

CASSP coordinator on the State Interagency Advisory Council for Systems of Care (SIACSOC). 
Additionally, CASSP launched and supported the development of a formal family advocacy capacity in 
California by providing focused, ongoing assistance to parents involved with creating and growing the 
Family Network of California (FNC) and the United Advocates for Children of California (UACC) (5). 
CASSP also created a state-wide Cultural Competence Advisory Committee (CCAC) aimed at 
improving the cultural competence of service delivery to California youth and their families. 
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Conclusions and Implications 

In summary, despite significant challenges and obstacles to reform, a human service reform movement is 
underway in California; this movement has been inspired and supported by the national movement for 
integrated systems of care as exemplified by the Child and Adolescent Service System Program of the 
National Institute of Mental Health. The reform movement gained ground steadily in California during 
CASSP years, as evidenced by system of care expansion, mental health policy and financing changes, 
and increased leadership and activism on behalf of children, youth, and their families. Three thematic 
findings emerge from the review of documents: 

• There is a striking degree of consistency and agreement across sources about the identification of 
problems and level of progress achieved in reforming human services to meet the needs of 
California youth with serious emotional disturbance; 

• Human service reform is uneven across California counties, with state-funded, integrated system 
of care counties existing alongside counties apparently uninvolved in reform efforts; and 

• CASSP is a positive and constructive influence on human service philosophy, policy, and 
programs in California. Document authors emphasize CASSP contributions with respect to 
disseminating system of care principles and philosophy, and in fostering parent advocacy and 
multi-cultural involvement in mental health planning and policy making. State memoranda and 
reports document CASSP efforts and activities throughout California. County administrator 
reports and monographs reflect the positive role CASSP plays in local system of care development 
and enhancement. Parent advocates note the positive role CASSP serves in support of advocacy 
efforts. 
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8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 



Issues in Implementing a Randomly Assigned Study Design in a 
Community Setting: Lessons from the Vanderbilt/Stark County 
Evaluation Project 



Introduction and Program Description 

As the first study conducted to examine the clinical outcomes of a publicly funded system of care using 
a randomly assigned sample, the Stark County Evaluation Project encountered numerous 
methodological challenges. The purpose of this summary is to describe the process of implementing the 
study design, as well as to discuss the various decision-making mechanisms utilized and lessons learned 



To date, there is no scientific evidence that the expanded care system is more clinically effective than 
traditional services. The recently completed Fort Bragg evaluation found that a continuum of care was 
not more effective than a traditional insurance system for milita^ dependent children (Bickman, 

Guthrie, Foster, Summerfelt, Breda, & Heflinger, 1995). The primary aim of the research project was to 
assess the extent to which an expanded system of individualized care leads to improvements in the 
clinical functioning of children with severe emotional disturbance and their families. This 
research/demonstration was distinctive in that it examined the effects of change in an expanded system 
of care, in contrast to studying changes in individual components of the system (e.g., case management, 
intensive home treatment) and did not study a specially designed demonstration project but services 
provided without special funding. The site chosen for the study was Stark County, Ohio, a county which 
has several years experience of collaborative efforts in implementing a coordinated system of care. 

The Stark County project focused on two key questions: 

• What are the outcomes of an expanded system of care on the clinical functioning of children with 
severe emotional disturbance and on the functioning of their families? 

• What are the individual child and family factors that influence the outcomes? 

This summary of the Vanderbilt/Stark County Project will address process issues critical to 
implementing the research design. Outcome data will be published when the study is completed. 
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Method 

Criteria for Inclusion 

Participants were 360 children and adolescents (Stark Coimty residents) who are between the ages of 5 
and 17.5 years inclusive, with indications of Severe Emotional Disorder (SED), and who were involved 
in at least one child-serving system (Department of Human Services (DHS), Education, or Juvenile 
Justice). Participants were recruited through several intake points with DHS and Child and Adolescent 
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Service Center (C&A), the child-serving community mental health agency. Children referred from DHS 
were screened by C&A and met intake criteria for C&A expanded services before a child was 
determined eligible for the study. 

Random Assignment 

A unique aspect of this study was the random assignment of children. Study participants were randomly 
assigned to receive the expanded services through C&A or standard care from community agencies 
outside of C&A. The random assignment procedure was felt to be the most valid scientific design that 
would enable us to determine if the expanded services were effective. Under this procedure, all 
participating children had an equal chance to receive the expanded services. Families were told about the 
two treatment conditions during the informed consent process, but they did not know to which condition 
their child was assigned until after the initial study interview. The study was designed to ensure that 
participants remained in the group to which they were assigned until data collection was completed. 

Data Collection 

The interviews with parents, foster parents, children and adolescents were conducted initially, and every 
six months for three waves of data collection. Participants received a combination 
of computerized, written, and, at follow-up, telephone interviews. The baseline data collection included 
self-administered computerized and written interviews. During follow-up interviews, all information was 
collected by phone or mail. 
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Lessons Learned 

Cooperative management was accomplished through a steering committee composed of representatives 
from Vanderbilt, Mental Health, and Department of Human Services. Monthly meetings were held to 
identify problems, air concerns, sort out turf issues, and reach decisions on how to address concerns. 
Good communication and the ability to remain flexible were key elements in project management. 
Identification of roles for each organization was crucial for implementation. Development of procedural 
protocols was also important. 

Vanderbilt played the role of providing data regarding recruitment, documenting services provided to 
clients, and encouraging the maintenance of standards required by the research design. Mental Health 
and Department of Human Services used the monthly meetings to continually coordinate their efforts at 
identifying untapped pools of potential participants, and to develop procedures for accessing those 
potential participants for the study. 

During implementation, barriers to success of the project were identified and addressed by the steering 
committee. The major barriers pertained to ethical concerns and participant recruitment. 

Ethical Concerns 

In implementing the research project with random assignment, several ethical questions had to be 
addressed. Those questions were raised by direct care providers in an effort to assure themselves that 
they were in compliance with their professional ethical standards. The ethical concerns raised were as 
follows: 

1. Is it ethical to deny the control group access to a system of care that providers believe to be the 
best way of providing service to families? 

2. In such a study, is it ethical and/or practical to ask C&A therapists and case managers to provide 
limited service to control group subjects? That is, should the project attempt to treat the control 
group within the system but withhold the expanded services? 

3. Should experimental group clients be required to see only system of care therapists for services? Is 
it possible that a case manager could coordinate the service plan with a private therapist or 
provider from another agency? 



:Er|c 



G1 



12/10/98 12:10 PM 



8th.53 



http://lumpy.finhi.usf.edu/CFSroot/rtc/proceeding8th/8th.53.html 



4. If a control group client decided that they want to seek system of care services, can we deny 
access to those services? 

The steering committee used the Ethical Principles of Psychologists and Code of Conduct (AP A, 1 992) 
as their guide in addressing these questions. 

In addressing the first ethical concern, the steering committee debated the issue of professional opinion 
versus existing research findings. In his role as principle investigator, the Director of the project 
provided information substantiating the fact that no experimental studies have been conducted on this 
matter. It was the opinion of the steering committee that these were indeed professional opinions and not 
matters of fact. Therefore, there was no ethical bind for providers of service to participate in the study 
with knowledge that control group would not have access to the system of care. 

In regard to the second question, it was decided by the steering committee that it was not practical to ask 
system of care (experimental group) therapists to provide services to both the experimental group and 
control group subjects. Moreover, based on ethical issues, it was important to C&A that control group 
participants not be clients in their treatment system. By serving them outside of C&A, they were not in 
the role of denying services. 

Regarding question number three, it was decided that it was possible for a case manager from an 
experimental group participant to coordinate services vvdth providers from any agency. This is indeed the 
work they do on a regular basis. Therefore, it was not considered to be an undue burden or an ethical 
concern for case managers to work with system of care therapists and/or therapists from other agencies 
or private practice. 

Regarding question number four, it was decided that if a control group subject decided to seek services 
from the system of care that they are in effect, dropping out of the study. However, detailed procedures 
were developed to assist those clients in finding oAer satisfactory providers that are not part of the 
system of care. By providing this assistance to clients who were not satisfied with their current provider, 
it allowed the study to maintain subjects in a more true control group setting as opposed to mixing part 
of their treatment with system of care therapists. It would have been clearly unethical to simply deny the 
client's request for the sake of the study. Once the client had dropped out however, they may request any 
service without restriction. 



Subject Recruitment 

Recruitment of subjects emerged as a difficult issue in the study. Mental health did not have the volume 
of clients to supply 360 subjects within the specified time frame of the research project. Department of 
Human Services (DHS) had the volume of clients, but lacked the clinical expertise to judge 
appropriateness of referrals for the study. Juvenile Court had problems similar to DHS. Schools had the 
volume, but were hesitant to engage in experimental research because of potential public relations 
problems. 

In the end it required a collaborative effort between Mental Health and Department of Human Services 
with much encouragement from Vanderbilt to recruit enough participants for the study. That 
collaborative effort consisted of Mental Health providing consultation in the form of screening potential 
referrals from DHS before the referral was actually made to Vanderbilt. This assiored appropriateness of 
participants referred for the study and increased the volume to a level that would allow us to reach 360 
subjects within the time frame of the study. Other procediores were developed to recruit from juvenile 
justice as well as from local school systems. These efforts, however, were largely unsuccessful. 

Finally, procediores were developed for recruiting from area United Way agencies. These efforts were 
initiated by Mental Health because of their existing relationship with United Way. After initiating the 
dialogue, tihe Vanderbilt staff then followed up and designed procedures that would allow them to 
receive referrals directly from United Way agencies. Because of the amount of time involved in working 
out such collaborative procedures, it placed significant pressure upon Vanderbilt as the primary 
investigator in this project to maintain the study within the time frame specified with the National 
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Institute of Mental Health. 
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Summary 

Collaboration between Vanderbilt, Mental Health and the Department of Human Services was crucial in 
addressing methodological and procedural issues to maintain the experimental design. Future 
community-based research efforts would be well advised to establish a steering committee to assure 
good communication and successful implementation of research design. 

The Vanderbilt/Stark County Evaluation Project is the first study to examine clinical outcomes in a 
publicly funded system of care using random assignment of subjects. The study has two key questions. 
One was to evaluate what the outcomes of a systemic approach to care are regarding clinical functioning 
of children with SED and their families. A second question was to look at what individual and family 
factors influence outcomes. 
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Background 

For the past decade, the pendulum for children and adolescent mental health services in Region II in 
Louisiana has been swinging from traditional institutional care to services that are child-centered, family 
and community-based. As the pendulum has swung, a continuum of services has developed that allows 
families of children and adolescents with severe emotional disturbances to choose from an array of 
services ranging from traditional inpatient care to home-based services. A goal of the developing system 
is to become a "one-stop" and "user friendly" system of care. This paper describes our system of care 
and all of its service options. Further, this paper describes a plan that is being developed to collect data 
from the system so as to objectively and scientifically study the effectiveness of the system as a whole 
and its individual parts. 

Region II of the Office of Mental Health in Louisiana serves seven parishes. The Margaret Dumas 
Mental Health Center (MDMHC) serves children ages 6 to 1 8 in Baton Rouge and the surrounding 
parishes. East Baton Rouge Parish is the largest urban area in the region with the outlying parishes being 
mainly rural. There are over 8000 children with severe emotional disturbances under the age of 18 in this 
seven parish area. This figure is based on an estimated five percent of the population. Many of the 
children in the area are served through the psychosocial rehabilitation option plan funded by Medicaid. 
Those served through the MDMHC have access to the continuum of care discussed below. The majority 
of cases at MDMHC have been of the lower socio-economic status and uninsured. 

Return to Top 

The System of Care 

On a continuum from least restrictive to most restrictive, our current system of care includes family 
support groups, educational groups, traditional outpatient counseling, home and school-based services, 
case management services, respite services, home companions, school companions, intensive 
home-based crisis counseling, runaway watch, suicide watch, therapeutic foster care, day treatment and 
inpatient treatment. A child or adolescent may move in either direction on this continuum at any time 
during his/her treatment. Each treatment plan is tailored to meet the individual's and family's specific 
needs. The following are descriptions of each treatment option. 

Family Support Groups: A local chapter of the Louisiana Federation of Families for Children's Mental 
Health is operated through the Margaret Dumas Mental Health Center and meets regularly to provide 
support for families of emotionally disturbed children/adolescents. They also maintain a resource library 
which contains information pertaining to mental illnesses and other relevant topics. 

Educational Groups: Currently providing three educational groups: Attention-deficit Hyperactivity 
Disorder, Anger Management, and Parenting Skills. Open to children and families receiving mental 
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health services. 

Traditional Outpatient Services: Traditional counseling and medication services at the mental health 
center by appointment or in emergency situations. Three social workers, one psychologist, and three 
part-time psychiatrists provide services. 

Home and School-Based Services: Traditional counseling services, but within a more natural setting, 
such as the child/adolescent's home and school. 

Case Management: Assistance with a variety of physical, social, and recreational needs of the child and 
family; assistance includes coordination of services received and an advocacy role. 

Brief in-home or out-of-home services: Supervision of the child/adolescent is provided by 
paraprofessionals. Allows primary caregivers to have time off from supervisory responsibilities. 

Includes both planned and emergency respite. 

Home Companion: A paraprofessional maintains one-on-one contact with the child/adolescent during a 
specified period of time within the home. Assists with supervisory duties to relieve primary caregivers. 

School Companion: A paraprofessional maintains one-on-one contact with the child/adolescent during a 
specified period of time within the school. Assists teacher in keeping the child/adolescent on task and in 
maintaining self-control. 

Intensive Home-Based Crisis Counseling: A four to six week program of intensive home-based 
counseling to prevent out-of-home placement of the child/adolescent. The child/adolescent must be at 
moderate risk for out-of-home placement within the next few months to qualify. 

Runaway Watch: 24 hour, one-on-one coverage for the child/adolescent who is a significant threat to 
runaway from home. Allows primary caregivers to sleep at night, etc. Attempts to prevent out-of-home 
placement of the child, and is an alternative to hospitalization. 

Suicide Watch: 24 hour, one-on-one coverage for the child/adolescent who is a significant threat to 
commit suicide. Allows primary caregivers to sleep at night, etc. Attempts to prevent out-of-home 
placement of child and is an alternative to hospitalization. 

Therapeutic Foster Care: Temporary placement of the child/adolescent with a foster family for 
up to four months while problems with family or environment are resolved. An alternative to 
hospitalization. 

Day Treatment: Mental Health Services provided on a daily basis within a structured setting. However, 
the child/adolescent returns home each night. An alternative to hospitalization. 

Inpatient Treatment: Hospitalization for intensive inpatient services. Most restrictive of all options. 
Return to Top 



Barriers 



While the system of care provides many treatment options, it is limited in its resources. Inherent in the 
task of developing a system of care for the children in Region II is the need to create a stable, long-term 
financing mechanism to support both the development and the operation of such systems. Presently in 
Louisiana, there is not a special children's mental health budget. All funds over annually budgeted means 
of financing generated by either adult mental health or children's mental health services are returned to 
the State General Fund. This mechanism not only handicaps the day-to-day operation of the mental 
health system but also limits the possibilities of pursuing Federal Grants. On many occasions, the State 
was not able to meet the grant requirement of coming up with "matching funds." A creative way of 
generating and managing a special fund for the system of care is needed. Due to these funding 
restrictions, resources have to be monitored carefully. Therefore, it is imperative that the components of 
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the system be evaluated so that the most effective components are available to the most people in need. 
We are in the process of developing a program to evaluate the effectiveness of the system. 

In addition to the barriers presented by funding problems, there are also transportation problems for the 
clients and a need for more clinical and clerical staff. Other aspects of the program that we would like to 
see added to the continuum of care include an after school program and more recreational services. 

Data being monitored include broad demographic characteristics to allow studies on urban vs. rural 
factors, age, racial, gender, and economic fkctors, and individual school factors. Other data collected 
include diagnostic information, medications, service utilization (least restrictive to most restrictive), 
hospitalizations, and other treatment factors. These data will allow us to formally evaluate effectiveness 
and to perform studies aimed at deriving predictor variables which will help plan our system more 
efficiently. If we can determine specific characteristics of clients that predict success or failure within a 
specific component of the system, we will be able to more efficiently use the limited resources we have. 
Return to Top 
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Introduction/Purpose 

This report provides a historical review of changes in Arizona's behavioral health system which occurred 
during the past four years. These changes were designed to develop public sector Managed Behavioral 
Healthcare Organizations for children and adolescents. Using managed care principles, the use of costly 
inpatient/institutional care was decreased, while commvmity-based outpatient care increased. A review of 
service utilization data is included which supports the transition from a fragmented fee-for-services 
system of care to the current level of system development. 
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Method/Procedures 

This project involved the review of historical documents, including state and federal legislation, state 
plans and waivers, intergovernmental policies and agreements, reports, memos, and contracts which 
tracked the multi-year system changes within the state's behavioral health system. As these changes were 
implemented over a four year period, client and service data were collected which described utilization 
under the previous fee-for-service system. These data were compared to client and service data reported 
since the development of the current managed care system. 

Subjects/Sites 

Client and service data from the four year transition period were reviewed for all Arizona children 
presenting with behavioral health needs, regardless of income and level of need. Data for this review 
period were selected from the state's management information system, selecting those children receiving 
at least one state-funded behavioral health service through contracted commvmity-based regional 
behavioral health authorities or their subcontracted providers. 

Intervention/Program 

Six Regional Behavioral Health Authorities (RBHAs) serving children and families in Arizona are 
located throughout the state. These are private non-profit corporations which receive funds through 
contracts with the state to administer behavioral health programs. These RBHAs subcontract with local 
service providers, providing commvmity-based delivery systems within their designated geographical 
bovmdaries, creating provider networks. The RBHAs also provide a case management system designed 
to provide authorization and conduct utilization review. 

Arizona state statute 36-264 requires a Medicaid financial eligibility screening prior to the utilization of 
publicly funded behavioral health services by children and adolescents. Once this screening completed, 
the child behavioral health assessment. As a result of the assessment, a behavioral health treatment plan 
may be developed. 
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Approval for services is based on medical necessity. Medical Necessity stipulates (1) A child requires a 
particular type of service; and (2) the child will benefit from that service based upon diagnosis, 
presenting problems, intensity of service need, and expected response (outcome). For admission to 
inpatient or residential treatment centers and partial care programs, level of care criteria are imposed 
which also facilitate overall requirements of medical necessity and impose RBHA case management 
designed for control of utilization. The required RBHA Prior Authorization is determined by a clinical 
team of psychiatrist, case manager, and masters-level supervisor, and it applies for any child entering 
out-of-home care. 

Referral into the RBHA managed care system results in two distinct avenues, one for children entitled to 
behavioral health services under Medicaid, the other for those not meeting Medicaid entitlement, but 
meeting medical necessity eligibility. In addition to level-of-care evaluation to control utilization, 
intensive case management services are assigned to children with a diagnosis and presenting with a 
functional score indicating critical severity of need. The Children and Adolescent Functional 
Assessment Scale (CAFAS; Hodges, 1990) is used as the functional assessment instrument. Intensive 
case management acts as gatekeeper for restrictive services. RBHA intensive case management 
personnel work under the direction of board-certified psychiatrists, and develop an Individual Service 
Plan, addressing medical necessity, appropriateness of services, prior authorization, and service 
utilization issues. 

After a child is placed in inpatient or institutional care, ongoing utilization reviews 
are conducted by the RBHA to determine appropriateness of placement. At this point, the case manager, 
via the clinical team, may request continued placement based on medical necessity. If denied by the 
RBHA clinical review team, the child is then transferred to a less restrictive treatment environment, such 
as community-based therapeutic group home, behavioral health group home, group home, therapeutic or 
non-therapeutic foster care, or return to home. Each of these placements may be supported by optional 
outpatient services, ranging from partial care programs to home-based counseling. 

Measurement/Instruments 

Client and service data were reviewed for utilization patterns under the previous fee-for-service system. 
These data were then compared to client and service data to determine utilization trends under the 
current managed care system. During the four year period of review, initial data reflected utilization 
patterns for a mixed capitation fee-for-service system. RBHAs were paid a monthly capitation amount 
for case management, transportation and assessment services, while their subcontracted providers were 
paid on a fee-for-services basis at pre-set maximum capped rates. These rates were projected, using past 
utilization and market rates. Later, under a fully imple-mented managed care system in which RBHAs 
were paid a monthly capitation amount based on prior enrollment, assuming risk for all services in the 
continuum of care, data were also tracked for utilization patterns. In this scenario, RBHAs authorized 
services through the state's manage-ment information system, and their subcontracted providers 
presented claims for reimbursement by the state's third party payor. 
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Results 



In Figure 1. service utilization data depict the average days of all admissions to out-of-home services 
over the past four years. Figure 2 reports the number of children (unduplicated count) receiving 
out-of-home services. As seen in these Figures, the average number of days in an inpatient/RTC setting 
decreased while the overall number of children receiving this service increased. The increase in 
utilization seen in 1 992 is thought to be reflective of the state's policy to bring children from other state 
agencies into the Medicaid program, many of which enter the behavioral health system from an inpatient 
setting. 



The data for children in other out-of-home programs such as therapeutic group homes has shown a 
decrease during the review period as has the average days in these programs. It should be noted that 
these out-of-home services are provided in an "unbundled" fashion: a basic room-and-board service fee 
is billed separately, along with any required therapeutic services, and each component may be paid. 
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provided, and recorded separately. For example, the room-and-board cost may be paid by other funding 
sources such as a child welfare agency and may not be accounted for in the behavioral health 
management information system. Because of this, it may not be possible to capture the exact number of 
children receiving behavioral health services in a less restrictive out-of-home program. Therefore, some 
under-reporting of the number of children in these out-of-home services may be evident. Moreover, the 
standard deviations on average stay were unavailable, thus caution should be used in interpreting these 
results. 

Figure 3 reveals that the average number of days of non-residential services remained the same 
throughout the four year period. This is remarkable when one considers the number of children 
(unduplicated count) receiving these services /see Figure 4). The number of children receiving in-home 
services increased five-fold over the four year period, while those children receiving services through a 
partial care program doubled during the same period. 
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Implications 

Preliminary data suggest that Arizona's attempt to provide behavioral health care in a publicly 
administered environment has yielded positive results. Managed care principles in the public sector can 
be effective in reducing the utilization of inpatient/residential care for children and adolescents. 
Additionally, the increase in utilization of non-residential services may indicate that community-based 
services are a contributing factor in the management of behavioral health services for children and 
adolescents. 
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Introduction 

Proponents of the continuum of care treatment philosophy have argued that by expanding the array of 
treatment options for which insurance will pay and allowing mental health providers to use their best 
professional judgment to assign children to treatment cost per treated child will decline. The argument is 
premised on the observation that the intermediate services of the continuum of care cost less per unit 
than more restrictive settings (e.g., hospitals and residential treatment facilities); allowing providers to 
rely upon their expertise (via insurance coverage) will permit providers to substitute less expensive care 
that is equally, if not more, effective at ameliorating children's emotional and behavioral problems. 
Return to Top 



Method 

Data Source and Definition of Variables 

The data for this paper come from the Evaluation Sample of the Fort Bragg Evaluation Project (FBEP). 
The results are estimated from a sample of 982 children and adolescents for whom the first six months of 
treatment cost was calculated. 

The service utilization of children was grouped into four categories according to the restrictiveness of 
services received: outpatient (OP) only, in which children received none of the new intermediate 
services or hospitalization/RTC treatment; intermediate non-residential (INR) care, in which the client 
received at least one INR service, perhaps in combination with OP therapy, but never any service in a 
residential setting; intermediate residential (IR) care, in which a child received at least one day of 
treatment in an IR facility (e.g. a therapeutic group home), perhaps in combination with hospitalization, 
RTC, INR, or OP therapy, but not necessarily; and, hospitalization/RTC (H/RTC) care, in which the 
client received at least one day of treatment in a hospital or RTC, perhaps in combination with OP 
therapy, but never in combination with any of the intermediate services. This serves as the dependent 
variable for the multinomial logit model. The other controls for the mutinomial logit and quantile 
regressions are listed in Table 1. 
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Results 



Factors Predicting Use of Intermediate Services 

One might expect that any relationship between costs and the service use categories to be a function of 
the severity of children's conditions. If intermediate services cost more, it may be because they are used 
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for the children most in need of service. To account for this possibility, a multinomial logit model is 
estimated using the controls listed in Table 1. Indeed, based on the results of the multinomial logit 
model, severity is predictive across equations at both the Demonstration and Comparison sites, 
particularly for the IR care group. However, as may be seen in Table 2. many other factors predict use in 
the IR category, and very few predict for the other categories. Before one can conclude that differences 
in the costs at different quantiles are attributable to these severity differences, we explore the results of 
the quantile regressions. 

Quantile Regression Estimates 

Quantile regression is used to analyze cost effects because its estimates are robust to violations of the 
normal distribution in the dependent variable (as is frequently the case in health and mental health 
expenditures). The coefficients are directly interpretable as in dollars (one carmot exponentiate the 
marginal effects of an OLS that uses logged expenditures and get the effects in terms of dollars; see 
Duan, Manning, Morris & Newhouse, 1982, for why this is so), and it seems plausible that the effect of 
intermediate services on a child's costs will differ depending on where in the distribution that child falls. 

Table 3 presents the quantile estimates for the 25th, 50th, and 75th percentiles. Quantile regression was 
used to estimate the effects of intermediate services for children at the 25th, 50th (median), and 75th 
percentiles of the cost distribution. 

Important things to observe include: 

• the effect of IR services increases as one moves up the cost distribution; 

• at each quartile, moving a child from OP to IR would increase costs by three times what a move 
from OP to H/RTC would produce; 

• the effect of INR approaches that of H/RTC as one moves up the distribution (i.e., INR is cheaper, 
but not that much cheaper); 

• INR costs substantially more than OP; and 

• introduction of IR and INR did lower H/RTC relative to what was experienced at the Comparison 
site. 

Interestingly, service type rendered nearly all other controls&shyp;including previous 
utilization&shyp;irrelevant in explaining effects at each quantile for the Demonstration, whereas at the 
Comparison various diagnostic and family demographic characteristics were significant predictors of 
cost of the first six months of treatment. 

The null finding for severity at the Demonstration in the quantile regression, coupled with the 
significance of severity in the multinomial logit suggest that severity influences cost through the process 
of sorting children into levels of care, but has no direct effect on cost otherwise. 
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Conclusion 



The quantile regression estimates showed that introducing intermediate services produces different 
effects depending upon where in the cost distribution one lies. The increase in costs due to intermediate 
services are much greater than those of traditional inpatient and outpatient services. 

The major limitation of these results is the ability to attribute the effects observed exclusively to the new 
services at the Demonstration. These results are potentially confounded with changes in cost sharing for 
services that were provided free of charge to families at the Demonstration. For traditional services this 
amounted to a price reduction of 20%; for the new intermediate services, the reduction was 100%. Thus 
far the Evaluation has not been able to satisfactorily separate the price effects from the service-type 
effects; disentangling these effects wall be a major research agenda item over the next few years. 
Saunders (1996) applies a three-part model similar to the model employed by Duan, et al. (1982) for the 
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Rand Health Insurance Experiment. However, the quantile regression results are still important because 
they support the conclusion that relying exclusively upon clinical wisdom as to what is appropriate 
treatment will not produce cost savings in addition to not producing superior mental health outcomes 
(Bickman, et ah, 1995). 
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Introduction 

Recent changes in the delivery of mental health services to children and adolescents have been driven by 
the evolving notion of the continuum of care.(Stroul & Friedman, 1986). A cornerstone of such a system 
of care is the provision of a diverse range of mental health services. A full range of services are needed 
to accommodate the diversity of needs among children with emotional and behavioral disorders and their 
families. An added principle of the continuum of care is that when individuals are placed in services 
along the continuum they are placed in the least restrictive setting. 

Intermediate services&shyp;both residential and non-residentiaI&shyp;are a key component of the 
continuum of care. They represent services more restrictive than outpatient therapy, yet less restrictive 
than inpatient hospitalization or care in residential treatment centers. 

An added benefit of introducing these services is that they are generally less expensive than more 
restrictive services. The introduction of intermediate services may improve mental health outcomes 
while reducing costs. 

This summary examines one aspect of the introduction of intermediate services; in particular, it 
considers the effect on costs at the system level. The analyses are based on the experiences of the Fort 
Bragg Demonstration. Under this demonstration, services previously unavailable under the Civilian 
Health and Medical Program of the Uniformed Services (CHAMPUS) were made available to military 
dependent children and adolescents. Known as The Fort Bragg Evaluation, the full results of this study, 
are reported elsewhere (Bickman, et al., 1994). 

Under the Demonstration, a wide range of intermediate services were made available: in-home 
counseling, after-school group treatment services, day treatment services, therapeutic homes, specialized 
group homes and 24-hour crisis management teams. As described in Bickman, et al. (1994), the 
introduction of intermediate services was accompanied by other changes in the service system. Care was 
coordinated by treatment teams and by case managers. Perhaps most important for the study here, 
however, is that the Demonstration required no financial outlay for families; usual co-payments and 
deductibles were waived. The Demonstration was funded under a cost-reimbursement contract; 
therefore, (theoretically) no limits were placed on the types of services to be offered as long as they were 
therapeutically appropriate. The philosophy of the Demonstration called for controlling costs by 
providing a continuum of services designed to be appropriate for each child, rather than placing a limit 
(or cap) on services or cost per child. All results from the Demonstration should be interpreted in light of 
this fact, and we discuss the implications below. 
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Methods 

The Fort Bragg Evaluation focused on a systematic comparison of care at the Demonstration site with 
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care as usual. The latter was represented by two comparison sites, Fort Campbell and Fort Stewart. 

Taken together, these sites were of comparable size and had demographics similar to that for Fort Bragg. 
Care as usual refers to care under CHAMPUS, and coverage for mental health services under 
CHAMPUS is fairly generous. CHAMPUS has recently incorporated some elements of managed care. 
Hospital use now requires prior approval by Health Management Systems, the CHAMPUS utilization 
review manager. The analyses presented here focus on the utilization and cost data. 

For the comparison, these data were taken from the claims from the CHAMPUS system. The claims data 
include information on the type of services received and on expenditures for those services. Included 
were any claims filed between October 1, 1990 and September 30, 1993. (Fort Campbell provided some 
services outside the CHAMPUS system. Service utilization records were maintained in a management 
information system and from there were incorporated into utilization calculations.) 

Information on services received under the Demonstration was contained in a management information 
system. It described all services provided, including the date of service delivery, the specific service and 
number of units of that service received. Per-unit costs of services were calculated on the basis of 
budgetary information provided to the evaluation [Interested readers are referred to Bickman et al. 

(1994) for more details of this process]. 
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Analysis 

Why might one think that intermediate services would save money? Under the Demonstration, the cost 
of a day in intermediate residential care is 36% that of a day in an inpatient facility, while intermediate 
non-residential care is 50% (see Table 1). While the gap between care in a residential treatment center 
and intermediate services is smaller, the latter are clearly less expensive on a daily basis. This 
information alone, however, is not enough to guarantee that costs savings were realized. In particular, 
the intermediate services had to be used by significant proportions of children treated and they had to 
replace other, more expensive services. 

If intermediate services were used by only a small percentage of all children treated, it seems unlikely 
that the resulting changes would be evident at the system level. The first question, therefore, involves the 
extent to which intermediate services were used. 

Of all children treated at the Demonstration, 10.53% used intermediate services of some sort. 
Specifically, 5.7% of all children and adolescents treated during the Demonstration period received 
intermediate residential services; 8.2% received intermediate non-residential services (There was some 
overlap between the two; 59% of those receiving intermediate residential services received intermediate 
non-residential services as well). When combined, these numbers exceed the proportion of children who 
were hospitalized or were treated in a residential treatment center. 

When used, these services involved large expenditures. The average child treated in a non-residential 
intermediate setting received over $17,000 worth of services in that setting during the Demonstration 
period. The corresponding figure for residential services is $26,83 5&shyp;this exceeds the figure for 
inpatient hospitalization. 

Because significant numbers of children used intermediate services and because the expenditures 
involved were large, intermediate services represent a significant proportion of total expenditures at the 
Demonstration. Nearly 40% of total spending was on intermediate residential (19.50%) and 
non-residential (18.01%) services (see Table 2 ). 

The average child treated at the Demonstration received nearly $3,000 worth of intermediate services 
during the Demonstration period ($1,517 and $1,401 on residential and non-residential care, 
respectively). When combined, these figures exceed those for residential treatment centers and inpatient 
facilities. 



Six types of intermediate services were available: in-home services, after-school, latency partial 
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hospitalization, day treatment, therapeutic home, and out-patient therapy. Table 3 describes the use of 
each type. In-home services were used most ffequently&shyp;nearly half (46%) of the 638 children who 
received intermediate services used these services. Nearly a third of the individuals treated used day 
treatment. Latency partial hospitalization was the least used form of intermediate services. 

The sum of the figures in Table 3 exceeds one (1.67), which implies that children who used intermediate 
services used more than one type. This finding was not surprising; our earlier results indicated 
substantial overlap between the residential and non-residential categories. In sum, significant numbers of 
children treated at the Demonstration used intermediate services. A large portion of expenditures at the 
Demonstration went to these services, and if introducing them did not reduce costs, it was not because 
the services were unavailable. 

Whether this utilization lowered total costs, however, depended on the extent to which these services 
replaced other, more expensive services. As noted, a day of hospitalization was three times as expensive 
as a day of intermediate residential care. If, however, each day of hospitalization was replaced with 10 
days of intermediate residential care, the availability of less expensive services will not reduce costs. 

We examined the effect of the availability of intermediate services on total spending per client, including 
expenditures for hospitalization and care in residential treatment centers. Overall spending on 
hospitalization and on residential treatment centers was roughly 20% higher at the Demonstration site 
($1^436,300) than at the Comparison sites ($1 1,128,290; see Table 2). To some extent, this difference 
reflected differences in access. A different picture emerged, therefore, when we focused on expenditures 
per treated child. Here, we see that expenditures on hospitalization and on residential treatment centers 
was actually lower at the Demonstration ($2,227 vs. $4,003). 

Which perspective is valid, total expend-itures or expenditure per treated child? To the extent the 
availability of intermediate services lured children into the system, the added costs associated with new 
clients represent the costs of adding intermediate services to a system of care. On the other hand, if those 
costs are due solely to changes in cost-sharing requirements for parents, the reduction in the costs of 
hospitalization and of care in a residential treatment center per treated child may represent real savings. 

Even if we suppose that the reduction in expenditures on hospitalization and on residential treatment 
centers is real, the reduction is not large enough to offset the spending on intermediate services. The 
savings on those services ($1,776) is smaller than expenditures on intermediate services ($2,917). 
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Discussion 



Clearly, a day's use of intermediate services is less expensive than a day spent in a residential treatment 
center or in an inpatient facility. Any actual cost savings, however, depend on the way in which those 
services are used. This is more than a matter of sequencing (e.g., whether intermediate services are used 
before or after hospitalization). Rather, it is a matter of whether these services are substitutes or 
complements. 

To see this, one should think of Rumbaugh clinic as a factory that combines different inputs into the 
production of good mental health. In this case, the inputs include but are not limited to mental health 
services (e.g., family resources can be thought of as an added input). Economists think of inputs as being 
either substitutes or complements. Substitutes are used to replace each other&shyp;complements 
improve the productivity of each other and in doing so increase the demand for each. The key is whether 
the use of one input increases or reduces the demand for the other. 



Clearly, discussions of cost savings seem to presume that the intermediate services and hospitalization 
(or care in a residential treatment center) involve inputs that may be substituted one for the other. 
Further, it is assumed that they will be substituted at a rate less than their relative prices. If 
hospitalization is three times as expensive as intermediate services, then each day of hospitalization is 
replaced with three or fewer days of intermediate services. 
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It may be the case, however, that hospital-ization can be replaced only with several days of intermediate 
services. Indeed, intermediate services may be used as a substitute not for hospitalization but for 
outpatient therapy. Making intermediate services available may move individuals away from the most 
restrictive end of the continuum; however, it may move them away from the least restrictive end as well. 
The latter outcome actually may increase costs. 

Another alternative is that intermediate services may not be substitutes for hospitalization at all. Rather, 
they may be complements&shyp;they may change the use of hospitalization and in doing so may make 
it more effective. Some evidence exists that this is the case. The use of hospitalization was dramatically 
different at the Demonstration. As discussed in Bickman et al. (1994), children hospitalized at the 
Demonstration left sooner but were more likely to return in the future. This change in the use of 
hospitalization may have been beneficial, and intermediate services may have made this possible. 

What is striking about this discussion, however, is that adding more of one input (intermediate services) 
and improving the productivity of the other (hospitalization) implies increased production&shyp;i.e., 
better mental health. As discussed in Bickman et al. (1994), there is little evidence that children at the 
Demonstration showed greater improvement. 

Several caveats are in order, however. The impetus to substitute may have been stunted by the absence 
of any costs savings. The very substitution assumed may depend on the presence of incentives to do so. 
The absence of cost-sharing not only lowers the costs of services to decisionmakers&shyp;it makes them 
equal and so eliminates the cost advantages associated with intermediate services. 

Obversely, no reason exists to believe that incorporating cost-sharing by parents or scrutinizing spending 
at the Demonstration more carefully would reduce all spending proportionately &shyp;in other words, 
there is no reason, for example, to believe that the price elasticity of demand is equal across services. 

The reduction in the use of hospitalization, therefore, may have been more dramatic had parents and 
program administrators been faced with hard choices about how to allocate limited resources. 

A second caveat involves the limited experience of mental health professionals with intermediate 
services. Because of this inexperience, they may have been used inefficiently. At this point, we know 
only that the services were used. It may be that too many people used them or too few. It may be that the 
wrong people used them; assigning different individuals to intermediate services may have lead to 
greater effectiveness. The fact that the Fort Bragg experiment was ground-breaking makes it 
interesting&shyp;this same fact, however, may limit the conclusions that may be drawn from it. 

Savings on reduced use of hospitalization, therefore, may have been greater in a system with different 
financing and in a system where professionals had more experience with intermediate services. 
Nonetheless, intermediate services were not inexpensive. In fact, the costs savings needed to pay for 
those services may be greater than stated here. The Demonstration was clearly more expensive per 
treated child, due partially to a large increase in the use of outpatient therapy. While this difference no 
doubt reflects reduced cost sharing, it also may reflect the introduction of intermediate services. In 
particular, children treated in less restrictive environments may require more supervision in the 
community. Additional outpatient visits may be a requirement of increased reliance on intermediate 
services. 

In sum, while substituting intermediate services for hospitalization offers a hope of costs savings, it is 
not clear that such savings were realized at Fort Bragg. It is not clear whether these savings would have 
been realized had different financial incentives have been in place: the lack of any cost sharing by 
parents may have dulled incentives to substitute intermediate services for hospitalization. 

The discussion here raises clear conceptual issues. It is unclear whether hospitalization and intermediate 
services are substitutes or complements. It is not clear, in other words, whether intermediate services are 
a replacement for hospitalization or a means of making it more effective. The answer to this 
question&shyp;and to questions about potential costs savings&shyp;await further research on the 
components of care. 
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Introduction 

One of the primary influences in the past decade on children and adolescent mental health services has 
been the Child and Adolescent Service System Program (CASSP). CASSP has advocated a 
comprehensive system of services, organized into a coordinated network of care, that is child-centered 
and family-focused, with the needs of the individual dictating the services provided (Stroul & Friedman, 
1986). 

At the same time the clinically-oriented CASSP principles have gained widespread recognition and 
acceptance, the economic underpinnings of mental health services have undergone profound change. 
Managed care, in its various forms, is rapidly becoming the dominant mode by which mental health 
services are funded and coordinated. 

Hypothetically, it is not difficult to imagine possibilities for both agreement and conflict between the 
goals of CASSP and the goals of managed care. On one hand, CASSP shares with managed care the 
importance that is placed on integrated and coordinated services. On the other hand, the CASSP 
emphasis on individual needs and flexible funds could be perceived as being in conflict with cost 
containment&shyp;the primary goal of managed care initiatives. Given the impact that both CASSP and 
managed care have had, and will continue to have, on child and adolescent mental health services, it is 
surprising that the relationship between these two influential movements has not been given closer 
empirical examination. 

In this summary, we examine the relationship between some of the clinically-oriented concerns of 
CASSP and the economically-oriented concerns of managed care by focusing on decision-making in 
emergency mental health services. Because emergency mental health often occupies a central position in 
mental health services, it is an ideal focus for investigating systemic patterns. Referrals come from and 
dispositions are made to almost every other mental health service through emergency mental health. 
While issues of clinical decision-making for children and adolescents in acute crises have been discussed 
in the literature (e.g., Costello, Dulcan & Kalas, 1991; Hillard, Slomowitz & Deddens, 1988; Slagg, 
1993; Way, Evans & Banks, 1992), the role of fiscal and larger systems changes in decision-making has 
not yet been addressed. 

The Emergency Mental Health Services (EMHS) program of the University of Massachusetts Medical 
Center serves the entire central Massachusetts area, from New Hampshire to the Connecticut border, and 
includes the metropolitan Worcester area, as well as some rather rural areas of the state. EMHS is 
located directly adjacent to the medical emergency room. Its clients include individual adults and 
children who are either self-referred, referred directly from an outside agency, or who are transferred 
from the medical emergency room. Presenting problems range from suicidality to psychotic symptoms 
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and other disruptions in behavior. Occasionally, clients will present with violent or self-destructive 
behaviors and require either chemical or physical restraint. 

Clients are assessed by an EMHS clinician and an attending psychiatric resident. In some situations, a 
client will stabilize while at EMHS; after stabilizing, he or she is then sent home. In other cases, further 
intervention and treatment will be necessary, and the client will be referred to outpatient services, a 
community-based crisis stabilization program or will be admitted to an inpatient psychiatric setting. 

Data were obtained on EMHS dispositions for children and adolescents over a 2-1/2 year interval 
covering the period before and after the implementation of privately managed mental health benefits for 
Medicaid recipients. These data allow for clarification of the contributions of clinical, economic and 
systems factors on decision-making and dispositional outcomes for children and adolescents in crisis. 
Return to Top 



Method 

Data were drawn from EMHS log sheets that are completed daily and contain clinical and demographic 
information for all clients who receive services. Data for clients under 18 years of age were obtained for 
a 2-1/2 year period and divided into three different periods: the pre-managed care period 
(10/1/91-9/30/92); the transition period immediately following the implementation of private 
management of Medicaid mental health benefits (10/1/92-3/31/93); and the post-managed care period 
(4/1/93-3/31/94). 

During these 2-1/2 years, there were 877 episodes when children were seen at EMHS. This number 
denotes the number of contacts or encounters and not the number of different individuals seen at EMHS, 
While 78% of the individuals seen at EMHS used the service only once during this period, 22% did use 
the service more than once, with 3% using it more than 4 times. Since repeat usage is a typical part of 
emergency mental health services, we believe that focusing on episodes as our unit of analysis leads to 
the best representation of EMHS service provision. 

Encounters were pretty evenly divided between males and females, with females accounting for slightly 
more. Although the age of clients in these encounters ranged from 3 to 17, there was a significant skew 
to the right, indicating a proportionately high number of adolescents (mean ± SD = 14.3 ± -2.6; mode = 
16). Fifty-seven percent of EMHS child and adolescent episodes were covered by Medicaid, 21% were 
covered by an HMO, 14% were covered by private indemnity insurance, and 7% were uninsured. 

Of the 877 EMHS episodes included in this study, 25% resulted in inpatient admissions to either a 
public or a private psychiatric hospital. The largest portion of the remaining episodes (46% of the 
original sample) involved clients who were sent to home while the remainder went to short-term crisis 
stabilization, residential treatment or some other setting. 

As part of our univariate analysis, we compared two different variables. First, we compared the number 
of EMHS episodes in the pre-managed care period and the post-managed care period in order to 
determine if there were differences in the volume of child and adolescent episodes. Second, we 
compared the proportion of EMHS dispositions to inpatient settings in the pre- and post-periods to 
determine if there was a change in the likelihood of children and adolescents being hospitalized. 
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Results and Implications 

There was a significant increase in the number of child and adolescent EMHS episodes in the 
post-managed care period when compared with the pre-managed care period (episodes post, N = 393; 
episodes pre-, N = 297; Chi square = 13.36; p < .001). This is probably due, at least in part, to new 
regulations in the post-managed care period requiring that all Medicaid recipients referred for inpatient 
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mental health services be screened by EMHS. 

While there was an increase in the overall volume of child and adolescent episodes in the post-managed 
care period, there was a sharp drop in the proportion of inpatient dispositions (percent admitted to 
inpatient pre- = 31.7%; percent admitted to inpatient post- = 19.3%; Chi square = 14.626; p < .001). 

This finding has both economic and clinical implications. One of the explicit goals of the 

move to managed care is to decrease psychiatric hospitalizations since these services are by far the most 

expensive in the system of care. At least in this setting, this goal is being met. 

In addition to financial considerations, the CASSP principles established by Stroul and Friedman (1986) 
also argue that mental health treatment should be accomplished in the least restrictive setting possible. 
Since admissions to highly restrictive inpatient settings appear to have been diverted to less restrictive 
alternatives, this goal is also being realized. 

The next step for our project will be to consider the role that clinical factors play in determining 
inpatient admission in the managed care era. One of the fears of therapists and clinically-oriented 
researchers concerning managed care is that clinical considerations will be replaced by decision-making 
that is entirely driven by finances. Our data will allow us to further explore this possibility. 

Return to Top 



References 

Costello, A. J., Dulcan, M. K., & Kalas, R. (1991). A checklist of hospitalization criteria for use with 
children. Hospital and Community Psychiatry, 42(8) 823-828. 

Hillard, J. R, Slomowitz, M., & Deddens, J. (1988) Determinants of emergency psychiatric admission 
for adolescents and adults. American Journal of Psychiatry, 145(11), 

1416-1419. 

Slagg, N.B. (1993). Characteristics of emergency room patients that predict hospitalization or 
disposition to alternative treatments. Hospital and Community Psychiatry, 44(3), 252-256. 

Stroul, B. A. & Friedman, R. M. (1986). A system of care for severely emotionally disturbed children 
and youth. Washington, DC: Georgetown University Child Development Center, CASSP Technical 
Assistance Center. 

Way, B. B., Evans, M. E., & Banks, S. M. (1992). Factors predicting referral to inpatient of outpatient 
treatment from psychiatric emergency services. Hospital and Community Psychiatry, 43(7), 703-708. 
Return to Top 



Stephen C. Young, M.A. 
Research Assistant 

Loma Simon, M.A. 
Research Associate 



Joanne Nicholson, Ph.D. 

Assistant Professor of Psychiatry 
E-mail: jnicholson@Banyan.ummed.edu 

Department of Psychiatry 
Center for Psychosocial and 
Forensic Services Research 



Anne Bateman, R.N., Ed.D. 



O 

ERIC 



91 



12/10/98 12:30 PM 



8th.79 



http://lumpy.fnihi.usf.eciu/CFSroot/rtc/proceeding8th/8th.79.htmI 



Clinical Director 

Emergency Mental Health Services 
508/856-3562 

all authors: 

University of Massachusetts Medical Center 
55 Lake Avenue North 
Worcester, MA 01655 
508/856-5498 Fax: 508/754-5007 
Return to Top 



ERIC 



92 



12/10/98 12:30 PM 



8th.83 



http://lumpy.fnihi.usf.edu/CFSroot/rtc/proceeding8th/8th.83.html 



IHomel isearcm iinoexi 

8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

The Cost of Residential Schools for Children with Severe 
Emotional Disturbance 



Authors 



Introduction Methods Results Discussion References 
Back to Table of Contents 



Introduction 

Inpatient hospitalization for children with severe emotional disturbance (SED) is an expensive, 
resource-intensive treatment modality. This summary considers the costs of an alternative treatment 
setting, the residential school or treatment center which provides services to children with SED. The 
study examined the following questions: (1) What are the costs of residential schools that treat children 
with severe emotional disturbance? and (2) Does ownership or facility size affect the cost per child in the 
residential schools? 
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Methods 



Data for the study came from a national, stratified sample survey of day and residential schools for 
children with disabilities conducted by the Department of Education in 1988. Eligible facilities included 
those operated by education agencies as well as other state agencies or private organizations. Facilities 
were selected for inclusion if one of the following criteria were met: 



1 . the facility identified itself as primarily treating children with SED or autism or childhood 
schizophrenia, 

2. the facility reported that it treated a subset of children with SED, although not as their primary 
population treated, or 

3. the facility treated children with a non-physical handicap and SED as a secondary handicapping 
condition (i.e., definition of a secondary handicapping condition states that the child would be 
classified as handicapped under the secondary condition in absence of the primary handicapping 
condition.). 
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Results 

These residential schools represented significant expenditures on each child, although not as large as 
alternatives such as inpatient psychiatric hospital care, which can cost as much as $500 per day, or 
mental health residential treatment centers, which cost between $50,000 and $75,000 per year (Bums & 
Friedman, 1990). 

For residential programs, the average cost per child per year was $42,215 and for facilities with both 
residential and day (R&D) programs, the cost per child was $47,615 for residential students and $16,962 
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for day students. These costs implied a 2.9 billion dollar industry. Some medical, residential and 
educational costs incurred by the institutions might have been paid by different agencies and were not 
included in these estimates; therefore, the actual size of this industry might have been larger. 

The average facility size varied with type of ownership fsee Table IT Non-profit programs were smaller 
on average than either public or for-profit programs. For-profit schools had larger than average 
residential programs; 131 students on average, compared to 57 students overall in combined R&D 
facilities, and 110 students compared to 78 overall in residential-only facilities. Size of an institution is 
often considered an indicator of quality, with larger institutions offering lower quality. However, large 
size may also be associated with economies of scale, e.g., quality adjusted cost per case falls with the 
institute size. The relatively large differences in average size of the facilities by type of ownership may 
indicate that non-profit institutions are providing a higher level of quality. On the other hand, if there are 
economies of scale in terms of cost (or quality), the larger institutional size may actually indicate better 
economic performance. 

Ordinary least squares, regressing facility characteristics on the natural log of the cost per-child were 
used to examine the independent effects of ownership form and facility size on cost (see Table 2 ). The 
average teacher salaries were included to control for differences in cost of doing business by state. 

Ownership form did not have a significant effect on cost per-child, except for non-profit combined R&D 
programs. The non-profit programs were significantly more costly per child than the public programs. 
The coefficient on the non-profit ownership for the residential only facility was consistent with this 
result, but did not reach statistical significance. The number of children enrolled in the program was 
included to determine if there were any economies of scale. None of the coefficients on the size of the 
program were significant, which suggested that there were no significant economies or diseconomies of 
scale. 
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Conclusions 

From this brief look at residential schools for children with emotional disturbances we can make several 
observations. Cost per child did vary by ownership form, but was statistically significant only for 
non-profit R&D programs relative to public facilities. Residential services offered by publicly owned 
R&D facilities were more expensive than either for-profit or non-profit programs. Potential reasons for 
this cost difference may have been the case-mix or severity of illness of children served. There also is 
some evidence that public programs examined may treat a different population (see Table 3 ). For 
example, public programs reported providing treatment to proportionately more children with multiple 
disabilities than did the others, and were more likely to report the primary disability as something other 
than SED. 

Size of the facility did not appear to influence the cost per child. This result suggests that there are no 
economies or diseconomies of scale. 
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Introduction 

As capitation becomes the norm for the provision of publicly funded mental health services, it is 
increasingly important to have information about different population subgroups. Capitation will require 
astute providers to have an understanding of the different kinds of patients and payers they are serving. 
This research project explores the differences between patients funded by Medicaid and patients with 
commercial insurance in an adolescent and child psychiatric hospital. 
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Methodology 

We reviewed and selected data from the computerized client database at Cleo Wallace Center (CWC), a 
not-for-profit psychiatric facility for children and adolescents located in the Denver metropolitan area. 
Cleo Wallace Center serves 175 children and adolescents daily in three settings constituting a continuum 
of care&shyp;hospital, residential, and day treatment. For the purpose of this study, we selected child 
and adolescent patients who were discharged from the hospital program between November 1, 1993, and 
October 31, 1994. Patients are assigned to the child or adolescent inpatient programs based on age, 
diagnosis, maturity, risk factors, program design, and space availability. Typically, patients imder the 
age of 12 are placed in the child program and patients 12 years of age and older are placed in the 
adolescent program. For this study we accepted the program at discharge as the delineation between 
child and adolescent. We extracted a total of 732 patient records with either Medicaid or a commercial 
insurance company as their primary payer. Individual medical records were reviewed for any data that 
was missing in the initial extraction. 

Data was transferred from the computerized patient database to SPSS (Statistical Package for the Social 
Sciences) and analyzed. Key variables included length of stay in the hospital, number of re-admissions, 
diagnosis at discharge, gender, ethnicity, age at admission, where the patient was prior to placement, and 
where the patient went after treatment. For comparative purposes, only patients with an inpatient stay 
were reviewed. 

An 18% random sample of patients was identified for each funding type (41 with Medicaid, 44 with 
commercial insurance). Medical record reviews were done to collect information on the number and type 
of previous out-of-home placements and/or psychiatric treatments and medication usage before, during, 
and after treatment at CWC (see Table IT 
Return to Top 
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Length of Stay 

Patients with Medicaid funding (both adolescents and children) had longer lengths- 
of-stay than patients with commercial insurance (see Table 2 & FigurelT 

We reviewed an additional 132 adolescent and 43 child patients&shyp; not included in the previous 
analysis&shyp;who had commercial insurance and moved from the inpatient program to the day hospital 
program. Patients with commercial insurance (both adolescents and children) who moved from the 
inpatient program (average LOS = 6.6 days) to the day hospital program (average LOS for adolescents is 
7.3 days and for children 7.4 days) had a combined average LOS of 14 days. This was less than the 
average LOS for adolescents or children with Medicaid funding. 

Previous Psychiatric Experience 

Eighty-five patient charts were reviewed for previous psychiatric experience. Almost twice as many 
adolescents with Medicaid, versus adolescents with commercial insurance, had at least one previous 
psychiatric experience. In contrast, 54% percent of children with commercial insurance had at least one 
previous psychiatric experience compared to 36% of children with Medicaid. 

Diagnosis at Discharge 

Fifty-six percent of adolescents with Medicaid were diagnosed at discharge with a behavioral disorder 
compared to 35% of adolescents with commercial insurance. Fifty-five percent of adolescents with 
commercial insurance had an affective disorder diagnosis at discharge compared to 30% of adolescents 
with Medicaid funding. 

About half of the children with commercial insurance or Medicaid had a behavioral disorder at 
discharge. However, children with commercial insurance were more likely to have an affective disorder 
(39%) while children with Medicaid are more likely to have an anxiety disorder (26%). 

Before Placement 

Adolescents with commercial insurance were twice as likely as Medicaid patients to come from a 
psychiatric/medical hospital or from being on-the-run prior to admission at CWC. Adolescents with 
Medicaid were more likely to come from an out-of-home placement (foster home, group home, 
residential child care facility, shelter care) or from detention/police custody. 

Most children (93%) with commercial insurance came from home prior to placement at CWC. Only 
sixty-eight percent of children with Medicaid came from home; another 30% came from an out-of-home 
placement. 

After Treatment 

Seventy-eight percent of adolescents with commercial insurance returned home after treatment, while 
only 52% of Medicaid adolescents returned home. Thirty-three percent of Medicaid adolescents went to 
an out-of-home placement and 10% went to a more restrictive psychiatric setting like a state hospital. 
Only 3% of adolescents with commercial insurance moved to a more restrictive psychiatric setting. 
Almost all (93%) children with commercial insurance returned home. Seventy-four percent of children 
with Medicaid returned home after treatment, and 22% went to an out-of-home placement. 

Demographics 

• Average age at admission: Medicaid-funded patients discharged from the adolescent inpatient 
program were, on average, slightly older (15 years) than patients ^vith commercial insurance (14.3 
years). The average age for children discharged from the child inpatient program was the same (9 
years). 

• Adolescents were fairly evenly divided between male and female regardless of payer source. 

There tended to be more young male than female children regardless of payer source. 

• Ethnicity: About one third of patients with Medicaid (both adolescents and children) were 
non- White. Twenty-two percent of adolescents with commercial insurance and 12% of children 
with commercial insurance were Non-white. Based on Colorado public school enrollment data, 
74% of youth imder 1 8 years of age were Caucasian and 26% were from other non- White ethnic 
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groups. 



Medication Usage 

More patients with Medicaid were using psycho-tropic drugs at the time of admission than patients with 
commercial insurance: 45% of adolescents and 66% of children with Medicaid; and 30% of adolescents 
and 56% of children with commercial insurance. 

Seventy-seven percent of adolescents with Medicaid who were admitted on medications, 
had their medications changed while in the hospital compared to 100% of the adolescents with 
commercial insurance. Seventy-five percent of children with Medicaid who were admitted on 
medications had their medications changed while in the hospital compared to 83% of the children with 
commercial insurance. 

More adolescents and fewer children with Medicaid funding were discharged on medications than had 
been admitted while on medication. More patients with commercial insurance (both adolescents and 
children) were discharged on medications than admitted. 
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Summary 

Based on our study results, patients with Medicaid funding in a psychiatric hospital setting were 
different from patients with commercial insurance. 

• Patients with Medicaid funding had longer lengths-of-stay, and were more likely to have been 
diagnosed with behavioral disorders. 

• There were some race and age differences; adolescents with Medicaid funding were older, patients 
with Medicaid were more ethnically diverse than patients with commercial insurance. 

• Patients with Medicaid funding disproportionately came from out-of-home placements (such as 
group homes, foster homes, etc.) and detention, while adolescents with commercial insurance 
were twice as likely as patients with Medicaid to come from a psychiatric/medical hospital. 

• Patients (both children and adolescents) with commercial insurance returned home after treatment 
at a higher rate than Medicaid patients. 

• Both Medicaid and commercially insured patients appeared to use inpatient services appropriately 
for medication evaluation and stabilization. 

• Adolescents with Medicaid were more likely to have had previous psychiatric experiences than 
adolescents with commercial insurance. Children with commercial insurance are more likely than 
children with Medicaid to have had previous psychiatric treatment. 
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Introduction 

The purpose of the present study was to investigate the psychometric characteristics of the Child and 
Adolescent Satisfaction Questionnaire (CASQ) and the Family Satisfaction Questionnaire (FSQ) used in 
the evaluation component of the Texas Children's Mental Health Plan (TCMHP). The TCMHP is an 
interagency public mental health initiative to provide community-based services to children and 
adolescents with severe emotional and behavioral problems. The current study also sought to explore the 
nature of children's satisfaction with mental health services. 
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Method 

Instrument Development 

The FSQ and CASQ were developed by a team of evaluation personnel representing the nine agencies 
participating in the TCMHP. Items considered for inclusion in the measure were required to capture 
consumer perceptions of mental health services that are important to both the consumer and the goals of 
the TCMHP. The fourteen items that were ultimately chosen to appear on both questionnaires are related 
to general satisfaction with services, consumer relationship with the service provider, consumer 
participation, things learned in treatment, and general child and family functioning after services. 

Twelve items are in the form of a 5-point likert scale, one item is a checklist of barriers, and one item 
provides the opportimity for written comments. Only the twelve likert scale items are scored, yielding a 
total score of 12 - 60. 

Subjects 

Consumers who were eligible to participate in the study included children with severe emotional 
disturbance and their families who completed TCMHP services between September 1992 and January 
1994. Three hundred and fi% four parent questionnaires and 266 child questionnaires were returned. 

The characteristics of the children who returned the questionnaires were very similar to the total 
population of children served by TCMHP programs. Ninety-seven percent were between six and 18 
years old. Sixty five percent were male and 36% were female. Fifty-eight percent were White, 23% were 
Black, 1 8% were Hispanic, and 2% were from other ethnic backgrounds. 

Procedure 

Upon completion of services, parents and children were given a copy of FSQ and CASQ and two 
postage paid envelopes. They were instructed to complete the questionnaires independently and return 
them to the research office through the mail. 
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Results and Implications 

The mean FSQ score was 51 and the mean CASQ score was 50. There were no significant differences in 
mean satisfaction scores for variables such as ethnicity and service type for either measure. However, 
children and parents were more satisfied with services if the child was rated as completing services as 
compared to children who did not complete service. Cronbach's alpha, a measure of internal consistency, 
was .89 for the FSQ and .88 for the CASQ. Most items showed a moderate to high correlation with the 
total score for both measures. 
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A principal component factor analysis was performed to determine construct validity. Three factors were 
retained for both the FSQ and CASQ after a Promax oblique rotation was performed. Factor one, for 
both measures, was labeled "Treatment Effectiveness" representing high loadings for items related to 
treatment outcomes for the child and family, progress made in treatment, and things learned in treatment. 
Factor two, for both measures, was labeled "General Satisfaction" representing high loadings for items 
related to service provider characteristics, happiness with service, and willingness to return for services. 
The third factor for the FSQ was labeled "Family Participation" representing the very high loading of the 
family participation item. The third factor for the CASQ, labeled "Child and Provider Relationship", 
represented loadings of ftie items related to staff niceness, staff helpfulness and child participation. 

The FSQ and CASQ were correlated with other measures to determine concurrent validity. The 
correlation between the FSQ and CASQ was moderately high. The correlation between the FSQ and 
ratings of the service provider's satisfaction with the case was moderate while correlations with 
DSM-III-R GAF scores and ratings of treatment goals met were low. The correlations between the 
CASQ and the same measures were also low. 

The results of the study showed the FSQ and CASQ may be useful in the evaluation and monitoring of 
community-based mental health services. Both measures are internally consistent and seem to measure 
treatment effectiveness and general satisfaction with services as intended. The results also showed that 
children's perception of treatment outcome may be associated with the child-therapist relationship while 
the parent's perception of treatment outcome may be associated with general satisfaction with services. 
These results, taken together with die correlation of the FSQ with the service provider's satisfaction with 
the case, may indicate a very different experience between adults who seek and deliver services and the 
children who are the focus of mental health interventions. 

Return to Top 



Lawrence W. Rouse, Ph.D. 

Nancy MacCabe, Ph.D. 

Marcia G. Toprac, Ph.D. 

Research and Special Projects 

Texas Department of Mental Health and Mental Retardation 

P.O. Box 12668 

Austin, TX 7871 1-2668 

512/206-4817 

Return to Top 




107 



12/10/98 2:18 PM 



8th.97 



http;//lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.97.html 



I Home I I search 1 1 index I 

8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

Use of Consumer Satisfaction Information to Promote Change in 
Child/Adolescent Mental Health Settings 

Authors 



Introduction Methods Results Conclusions 
Return to Table of Contents 



Introduction 

The literature on assessment of satisfaction with child and adolescent mental health services is replete 
with instruments to assess adolescents' and parents' satisfaction, and the beginnings of a research base on 
factors that influence satisfaction is developing (reviewed in Young, Nicholson & Davis, 1995). While 
child and adolescent mental health agencies and research projects are strongly encouraged to gather 
information about consumer satisfaction, there is little guidance available in the literature on how to best 
use this information. For example, should a certain percentage of negative responses be used as a red 
flag to indicate that a program has a problem, or should all negative feedback be addressed? Additionally 
once it is determined that there is significant dissatisfaction, how should it be addressed? Should this be 
a standard part of the quality improvement process, should it lead to a focused assessment, or is there 
some other process by which the information can be used? 



The current study attempted to determine: (1) how consumer satisfaction information is most commonly 
obtained, (2) the purposes for gathering consumer satisfaction feedback, (3) the process for making the 
information useful, and (4) whether one of the barriers to utilization of consumer satisfaction feedback is 
the dismissal of its importance by professionals. 
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Methods 

Subjects 

Surveys, described below, were included in the packets of all attendees (approximately 500) of the 7th 
Annual Research Conference - A System of Care for Children's Mental Health: Expanding the Research 
Base, February 28 - March 2, 1994. This conference's participants include some of the preeminent 
researchers of children's mental health services, as well as service providers, administrators and 
consumers. Because of the low response rate, the survey was mailed to all attendees on the participant 
list (N=378). The total number of responses was 78. 

Instrument 

A written survey was developed that included a total of 9 questions, one of which contained several 
parts. Questions were open-ended (i.e., who provides satisfaction information?, why has consumer 
satisfaction been assessed?). Questions addressed characteristics of the respondent, methods used to 
assess satisfaction, the purpose of gathering satisfaction information and how the information had been 
used. 



Coding 

Because of the open-ended nature of the questions, answers were classified by creating categories using 
half of surveys, then these categories were applied to the other half of the surveys. The half on which 
categories were created was then coded again using the final categories to minimize drift. 
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Results 

Of the 78 respondents, 83% reported that they or their organization had assessed consumer satisfaction. 
The largest group of respondents were those from state or local agencies (36%), followed by direct 
providers (23%), respondents affiliated with universities, and respondents from some other setting 
(19%). Thirty-six percent of respondents had developed their assessment instrument, 31% had helped 
collect the data, 21% had been involved in the data analysis, 18% assisted in writing up the results, 8% 
participated in designing the methodology, and only 12% had no direct involvement in the assessment 
process. 

Assessment Methods 

Respondents described a variety of methods for gathering satisfaction information; however, written 
surveys administered to both parents and children/adolescents during or after treatment was the most 
common method ( see Table 1') Seventy-seven percent of respondents indicated that the identity of survey 
informants was consistently protected. 

Why has satisfaction been assessed/How has it been used? 

Respondents treated these two questions as the same question, indicating that the survey's intended use 
was the purpose for which it was gathered. Satisfaction information had been used for a variety of 
purposes, the most common of which was to guide program change, but also for guiding policy change, 
justification to funders, and adding to the literature on assessment of satisfaction. 

How has satisfaction feedback lead to changes? 

This question was intended to address the process by which the information lead to change. In 
retrospect, the question was too vaguely worded, and resulted in a reiteration of the changes that had 
been made rather than the process. However from the answers, it was clear that for 55% of respondents 
satisfaction feedback had been used to produce some specific change. Seventeen (22%) reported that the 
information had not been used, and another 22% described the process. In some cases, this process 
consisted of reporting the negative responses to boards or other groups that then decided upon action. In 
other cases, program directors personally contacted the respondents to address concerns not given 
anonymously. 

What are the advantages or disadvantages of collecting or using satisfaction information? 

The vast majority (81%) reported advantages of having consumer satisfaction information, the majority 
of which concerned matching program and policy practices with parent and consumer needs. Thirty-four 
percent cited disadvantages that included the labor intensity, cost of gathering the information, and the 
tendency of consumers/parents to rate services overly highly. 

Observed negative feedback being dismissed? 

Many respondents (46%) responded that they had never seen consumer feedback being dismissed, and 
several respondents expressed that negative feedback was the most useful kind of feedback. However, 
36% reported that they had seen information dismissed. Some respondents specifically stated that 
parents were perceived to be part of the child's 'problem,' and their opinions were invalidated because of 
this. Finally, others described professionals that simply felt that it was not valid 
information&shyp;usually for methodological reasons. 



Group Differences 

Comparisons were made between the responses of those who were university, direct service, local/state 
agency, or other based. The only methodological difference was in the time at which satisfaction was 
assessed, with university-based respondents reporting more baseline assessment (57% versus 26% for 
direct service providers, 1 1% for state/local agencies and 9% for other based respondents). As one might 
imagine, the purposes for gathering satisfaction differed between groups, \vith university based 
respondents the only ones assessing satisfaction for research purposes (23%), while agency-based 
individuals were more interested in knowing the level of satisfaction with their programs (15%, versus 
8% of university-based, 0% direct services, and 4% of other-based). How feedback was used also varied 
between groups. Ninety percent of those who were based in 'other' settings used the information to 
change programs (many of these were advocacy based settings). In comparison, this use was reported by 
63% of direct service providers, 33% of state/local agencies and only 21% of university-based 
respondents. Interestingly, 20% of direct service providers used the information for policy change, as did 




109 



12/10/98 2:19 PM 



8th.97 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.97.html 



7% of university based respondents, while none of the state/local agency or other-based respondents 
reported this use. 
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Conclusions 

In general, respondents were knowledgeable about procedures for assessing consumer satisfaction in the 
area of children's mental health. This should be considered a bias in the sample, and may not represent 
wide usage of consumer satisfaction assessment within children's service programs. While there was 
great variation in how satisfaction data was collected, there was a clear preference for written surveys 
assessed anonymously during and after treatment. It is encouraging that satisfaction was frequently, and 
equally, assessed for both parents and children/adolescents. 



The purpose and use of having consumer satisfaction feedback was varied and interesting. The primary 
purpose was reported to be to inform and improve services. Although the details of these service changes 
were not often presented, many respondents did report some changes being made in response to 
satisfaction data. At the same time, dismissal of negative consumer feedback was not uncommon. There 
were also clear differences in the reasons for collecting this type of data across the different service 
settings of the respondents. Lastly, the survey was not adequately constructed to assess the process by 
which satisfaction information leads to change. 



In our review of the consumer satisfaction literature (Young, et al., 1995), we found that procedures for 
utilizing consumer satisfaction feedback for programmatic or policy change were not addressed. With 
the current ubiquitous demand to obtain consumer satisfaction information, and the desire of policy 
makers, administrators and providers to know what is satisfying and dissatisfying about their services, it 
is important that concrete procedures to respond to consumer feedback be described and discussed in the 
future. 
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Purpose 

It has been conservatively estimated that 12% or 7.5 million of the nation's children suffer from mental 
health problems (National Institute of Medicine, 1989; U.S. Department of Health and Human Services, 
1990). Approximately 2 million of these children have received treatment, and about half of those 
receiving treatment have been getting inappropriate treatment (Saxe, Cross & Silverman, 1988). 

Services for these children have been described as fragmented, duplicated, and too restrictive (Knitzer, 
1982; Young, 1990). Services have also been criticized for not being community based, for being driven 
by the needs of the providers or payers rather than the needs of the children and their families, and for 
failing to include parents as part of the treatment team. 

The need exists to assure that appropriate treatment is available to children with mental health problems 
and to their families. An important part of this assurance is obtaining parent satisfaction information. 
Parent satisfaction scales are needed that are: (a) valid and reliable; (b) specific enough for meaningful 
interpretation (versus general, global statements); (c) based upon models with well-defined concepts; (d) 
actually measuring the concept, satisfaction; and (e) able to allow comparisons across studies and 
settings. 



The Parent Satisfaction Scale (PSS) and the Parent-Staff Interactions Scale (P-SIS), developed to 
address these needs, measure parents' perceptions of mental health staff providing services to their 
children. The PSS is made up of direct measures of parents' satisfaction with staff interactions. The 
P-SIS consists of parents' perceptions of staff interactions. The purpose of this summary is to describe 
the development, validity, and reliability of the PSS and P-SIS. 
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Methodology 

Instrument Development 

The literature related to consumer satisfaction and to mental health service delivery to children was 
reviewed, along with existing consumer and parent satisfaction scales, to identify items for the P-SIS. A 
total of 76 items were generated. Several conceptual sub-scales were identified, including staff attitudes, 
staff availability, informing parents, providing support, including parents, and staff helpfolness. 

Content and Face Validity 

Prior to content and face validity efforts, the 1 5 items in the conceptual subscale Helpfulness were 
deleted, as Lebow (1982) suggested, to avoid using outcomes from treatment as part of a satisfaction 
scale. The remaining 6 1 items were shared with two parents of children with mental health problems 
who were also directors of state-wide parent networks. They were asked to rate the items on a 4-point 
scale from extremely relevant to not relevant; to identify any content areas not included; and to make 
recommendations about the wording of items. Neither parent completed the rating scale, but did provide 
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face validity by making suggestions for additional items and wording changes. 

The items were also given to five professional experts in children's mental health and/or consumer 
satisfaction research. An 80% item support criterion was set to retain an item. All items except two met 
this criteria. With recommendations for wording changes, these two items were reworded and retained. 
This process established content validity. A total of 1 2 new items were recommended by the parents and 
professionals and added to the scale. With consultation from two doctorally-prepared experts in 
instrument development, the Helpfulness items were returned to the P-SIS because they appeared to be 
clear measures of parents' perceptions of how staff interacted with them. The P-SIS had 88 items. 

It was anticipated that the more positively parents perceived their experience with staff, the more 
positively they would rate their satisfaction with staff interactions. The 19 item PSS was developed by 
compiling direct questions about parent satisfaction for each of the conceptually derived P-SIS subscales 
along with an overall statement of satisfaction with staff to provide construct validity. Two independent 
items expected to be positively related to the P-SIS were also added for criterion validity including: a) If 
I needed services for my child again, I would go back to these staff; and b) I would recommend these 
staff to friends and neighbors with similar problems. All of the items were rated on a 5 -point Likert type 
scale from strongly agree to strongly disagree. 



Sample 

In the first of three waves of testing, the P-SIS (88 items) and the PSS (19 items) were tested with a 
convenience sample of Indiana support group members with 44 scales completed. After scale revision, 
the P-SIS (36 items) and the PSS (12 items) were tested with a convenience sample of parents of 
children discharged from a state, in-patient program, resulting in 24 completed scales. After further 
revision, the P-SIS (13 items) and the PSS (7 items) were distributed to 160 parents of students with 
mental health problems (in process at the time of this summary). 



Scale Revision 

The majority of items were deleted due to inter-item correlations above .70, indicating redundancy, with 
only a couple items deleted due to correlations below .30. The item-total correlation, alpha if deleted. 
Judgment of conceptual desirability, variance, and mean were used to further determine which of the 
highly correlated items to remove. 



To retain an item, it needed to have a high correlation to the total number of items, the alpha if deleted 
needed to be a significant decrease, the variance needed to be broad enough to show it was sensitive in 
discriminating the differences among individuals, and/or the mean needed to be relatively close to the 
midpoint of 3 to limit the positive skew seen in most satisfaction and perception scales (Lebow, 1982). 
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Findings 

Reliability and Criterion Related Validity 

A high inter-item correlation indicated high internal consistency and provided evidence for both scales' 
reliability with the alpha for both the P-SIS and the PSS being .98 (N=68). Criterion related validity was 
evident with the correlation of the P-SIS to independent would return and would refer items of .88 and 
.87, respectively (N=68). Further, the P-SIS was strongly correlated with the PSS (r=.98). 

Discussion 

The P-SIS and PSS were developed in response to the need to have parent satisfaction instruments with 
strong psychometric support for use in evaluating parents' satisfaction with services 
for their children with mental health problems. Validity and reliability of the P-SIS and the PSS were 
supported. The P-SIS and PSS, having been drawn from a large pool of items, provide specific enough 
information about parents' perceptions of staff interactions to be meaningful. Clustering of the item 
means around the mid-point of the 5 -point Likert scale and broad standard deviations indicate that the 
scales capture variability between subjects and do not have the usual positively skewed results seen in 
most satisfaction measures. These scales should prove useful in gaining information about parents' 
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satisfaction with and perceptions of staff interactions. 
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Introduction 

Family involvement in the provision of services for youth with serious emotional disturbance (SED) is 
an important correlate of family empowerment and provides an index of the service delivery system's 
responsiveness to the needs of families (Able-Boone, Sandall, Loughry, & Frederick, 1990; Collins & 
Collins, 1990; Dunst & Trivette, 1988; Zimmerman & Rappaport, 1988). Recent empirical work 
investigating the components of collaboration between parents and mental health professionals revealed 
family involvement to be of paramount importance in a successful collaborative relationship between 
parents and professionals (DeChillo, Koren, & Schultze, 1994). Various components of family 
involvement have been identified in the context of mental health service delivery, including: (a) the 
continuous involvement of the family in services, (b) appropriate education of the family about services, 
(c) involvement of the family in all decisions related to treatment, (d) encouraging the family to 
participate in all phases of the treatment process, and (e) keeping the family updated and informed on the 
process and progress of treatment (Able-Boone et al., 1990; Brotherson & Goldstein, 1992; Collins & 
Collins, 1990; Goodman & Hover, 1992). 

There is currently no simple, valid, and reliable rating scale that can be used to measure family 
involvement in the provision of mental health services. Given the importance of family involvement in 
the provision of services, its measurement and quantification would be a useful step in understanding the 
nature of the interplay between parents of children with SED and service providers. To this end, the 
current study examined the psychometric properties of a rating scale designed to measure perceptions of 
family involvement. The rationale for the scale was based on the concept of empowerment and the 
assumption that family involvement is intimately linked to empowerment, with more empowered 
families exhibiting greater involvement in the design and delivery of services for their children with 
disabilities (Koren, DeChillo, & Friesen, 1992; Singh et al., 1995). 
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Method 

Scale Development 

The initial pool of items was obtained through the critical incident method by asking families in focus 
groups what issues were important to them in the design and delivery of treatment services. These issues 
were phrased as individual items that could be rated on a Likert scale. Further items were generated that 
were commensurate with the general orientation of family involvement in the assessment, planning, and 
treatment of youth with SED. These items were then inspected for conceptual overlap, clarity, and face 
validity. Thirty items were retained for the final rating scale, with a 5-point Likert rating (1 = strongly 
disagree to 5 = strongly agree). Two parallel versions of the scale were developed: Perceptions of Family 
Involvement: Family Version and Perceptions of Family Involvement: Professional Version. The 
content of the items is identical between the two forms, while the wording of the items is appropriate for 
parents or professionals. 
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Participants 

The Perceptions of Family Involvement: Family Version was completed by 225 parents of children with 
SED and/or ADHD. Approximately 88% of the parents were mothers, 77% were Caucasian, and 17% 
were African-American. Most of the identified children were boys (71%), and 61% of the respondents 
were married. Respondents were evenly distributed among educational and income levels. The mean age 
of the parents was 41.5 (SD = 8.6) years, and the mean age of the identified child was 13.8 (SD = 4.4) 
years. 

The Perceptions of Family Involvement: Professional Version was completed by 264 mental health 
professionals. Approximately 40% of the professionals were men, 91% were Caucasian and 5% were 
African-American, and a majority identified themselves as either counselors or social workers. The 
modal professional had worked for 1 1 to 1 5 years with youths identified as having SED . The mean age 
of the professionals was 41.1 (SD = 9.1) years. 

Procedure 

The Perceptions of Family Involvement: Family Version was mailed to a sample of parents of children 
with SED. Parents completed the rating scale and a cover sheet which solicited demographic 
information. The Perceptions of Family Involvement: Professional Version was mailed to a national 
sample of mental health professionals who work with children with SED and their families. 

Professionals completed the rating scale as well as a cover sheet with demographic information. 
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Results 

A factor analysis of both versions of the Family Involvement Scale was undertaken to determine the 
underlying factor structure and the psychometric properties of the scales. A principal components 
procedure was used for the extraction of factors, with unity (I's) on the diagonal of the correlation matrix 
for deriving the initial estimates of communality. A varimax (orthogonal) rotation was used to obtain the 
rotated factor structures. Determination of the number of factors was guided by both statistical and 
conceptual approaches. Rather than strictly adhering to the eigen value greater than one rule or some 
other solely mathematical approach for determination of the number of factors to retain (Zwick & 
Velicer, 1986), solutions of various numbers of factors were examined with the following questions in 
mind: (a) did each factor make sense conceptually?; (b) was each factor a logical component of the 
concept of family involvement?; (c) did each factor appear unidimensional without being trivial?; (d) 
was the solution as a whole a parsimonious representation of family involvement?; and (e) did the factor 
structure have robust statistical properties? 

Perceptions of Family Involvement: 

Family Version 

Three- and four-factor solutions of this version were the most conceptually meaningful and statistically 
sound. Inspection of the scree plot (Cattell, 1978) and the coefficient alpha reliabilities indicated that 
either of these solutions would be statistically acceptable. However, the fourth factor of the four-factor 
solution contained three items that did not appear to fit together conceptually. In the three-factor solution 
two items (17 and 18) did not have a statistically significant loading on any of the factors. The resulting 
three-factor solution without items 17 and 18 in the analysis was determined to be the most conceptually 
meaningful and statistically robust. 

The three factors together accounted for 66.4% of the total variance. Table 1 presents relevant statistical 
properties of the factors and their coefficient alpha internal reliabilities. A factor loading of at least 0.40 
was used as the cut-off for a significant factor loading of an item. All but two of the items (17 and 1 8) 
loaded on at least one factor, and 12 items loaded significantly on two factors. The mean factor loadings 
for items on Factors I, II, and III were 0.69, 0.63, and 0.67, respectively. 

Factor I (Treatment) consisted of 14 items that generally represented family involvement in direct 
treatment services, including items about treatment satisfaction, follow-up, and treatment planning. 
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Factor II (Parent-Professional Partnership) consisted of seven items that represented decision making 
and planning about treatment as well as assessment in partnership with the family. Factor III 
(Information/Courtesy) consisted of seven items that represented keeping the family informed about the 
child's treatment, and showing respect and courtesy to the family in the treatment process. 

The Guttman split-half estimate of reliability was 0.91 for the final scale. The alpha coefficients for the 
factors are shovm in Table 1. These internal reliability coefficients are large, and indicate substantial 
internal consistency (DeVellis, 1991). Table 2 p resents the Pearson correlations among the factors. The 
magnitudes of the correlation coefficients are moderately high and all are significant at the p<0.001 
level. These correlations would suggest that the three components of family involvement revealed by the 
factor analysis are interdependent. 

Perceptions of Family Involvement: Professional Version 

A five-factor solution of this scale was the most conceptually clear and statistically robust solution. 
Although Factor V contained only two items, these items loaded together on one factor regardless of the 
solution examined and it was felt that together they represented a conceptually valid factor. The 
statistical properties of these factors are shovm in Table 3. The five factors together accoimted for 61.9% 
of the total variance. Again, a factor loading of at least 0.40 was used as the cut-off for a significant 
factor loading of an item. All of the items loaded significantly on imique factors. The mean factor 
loadings for items on Factors I, II, III, IV, and V were 0.63, 0.65, 0.63, 0.70, and 0.75 respectively. 

Factor I (Treatment Planning) consisted of 10 items that represented family involvement in treatment 
planning, including items about involvement in treatment planning meetings and formulation of a 
treatment plan. Factor II (Follow-Up) consisted of eight items that addressed issues concerning 
follow-up after treatment has been completed. Factor III (Decision-Making) included six items that 
represented family involvement in decisions about meetings, assessment, and the types of services to be 
provided. Factor IV (Assessment) consisted of four items concerning family involvement in the 
assessment process. Factor V (Global Involvement) consisted of two global items that concerned family 
involvement in treatment. 

The Guttman split-half estimate of reliability was 0.85 for the entire scale. The alpha internal reliability 
coefficients for the five factors are shovm in Table 3. These coefficients are large, and indicate 
substantial internal -consistency (DeVellis, 1991). Table 4 presents the Pearson correlations among the 
five factors. The magnitudes of the correlation coefficients were moderately high, with the exception of 
the correlations of Factor V with the other factors, and all are significant at the p<0.05 level. These 
correlations would suggest that four of the components of family involvement revealed by the factor 
analysis are interdependent, while the Global Involvement factor only weakly correlates with the other 
four factors. 

Return to Top 



Implications 

The results of the factor analyses of the parallel forms of the Family Involvement Scale indicate that the 
concept of family involvement is a valid construct, and the instrument devised to measure it has robust 
psychometric properties. The two forms of the scale yielded somewhat different factor structures. The 
factors of the professional version paralleled the service delivery process more closely than the factors 
from the family version, which were more global. This is perhaps reflective of differences between 
professionals and families in the level of knowledge and perception of the mental health service delivery 
system. These scales will provide a useful tool for service providers wanting to assess the level of family 
involvement in services. Utilizing both forms will reveal discrepancies between professionals' and 
families' perceptions of family involvement, and will reveal areas where families perceive deficiencies in 
their involvement in services. 
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Introduction 

Respite care is the formal mechanism for providing short-term care to children with disabilities to 
provide relief to parents or other caregivers from the stress of child care. Respite care represents one 
highly effective mechanism in achieving the mission and policy of public mental health systems to 
reduce the consequences (frequency and intensity) of disability, impairment, disturbance, or illness for 
children, their families and their communities. Its beneficial effect on family functioning and crucial role 
in preserving families is well documented. It should be a prominent element in achieving the mission 
and policy of public mental health systems as they shift from institutionally based, pathologically 
focused, and professionally driven paradigms to community based, strength focused, consumer driven 
paradigms. 

Families report respite care to be an important source of support in their efforts to raise their children 
with special needs. The effects that families experience when using respite care are not uniform. This 
paper will examine the relationship among parents' perceptions of the impact of respite care, their level 
of social support, and intra-family resources' (money and time) availability on their stress. 
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Method 

Procedures 

A total of 3 19 families out of 889 families (35.9%) at nine county-based community mental health 
programs volunteered for the study. Interviews were conducted by telephone using trained interviewers 
and computer assisted interviewing technology with 162 randomly selected families. A follow-up survey 
was mailed to each parent who was interviewed with 150 returning the survey. This follow-up survey 
contained Friedrich's (1983) Short-form of the Questionnaire on Resources and Stress (52 items), 

Dunsfs (1988) Family Support Scale (19 items), and Family Resource Scale (31 items). 

The majority of families were white and had an average income between $20,000 and $29,999 fsee 
Table 1). Respondents were primarily female, i.e., mothers, with an average age of 44.13 years 
(SD = 13.31) and had completed high school or more schooling (89.5%). The largest percentage of 
respondents did not work outside of the home. Two-thirds of the interviewed families were two parent 
families. When compared to the service population of 889 families, the 162 interviewed families were 
not statistically significantly different in terms of ethnicity but did have higher income levels. 

The average age of the children (which included adult children) was 16.82 years (SD = 12.70). Slightly 
more than half the children were male fsee Table 2). Mental retardation was the most frequent primary 
diagnosis. The extent to which children's disabilities affected their functioning is presented in Table 3. 
Mental development, physical mobility, and ability to communicate were rated by the parents as 
somewhat to greatly affecting functioning. Parents assessed their children's need for assistance with 
every day activities as nearly total. They indicated that their children required specialized medical care 
several times per month and had major behavior problems at least once a week, on average. 
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Analysis 

Eight variables were selected to represent the major conceptual domains of a stress adaptation model 
(Singer & Irvin, 1991; Wilder, 1986) in the path analysis fsee Figure II. Seven of these domains served 
as independent variables: (1) presence of a spouse (0 = no, 1 = yes); (2) amount of care required by the 
child, calculated by summing the scores on the six functioning variables (a = .61); (3) perceived 
helpfolness of the parent's social support network as measured by the total scale score on the Family 
Support Scale low score = not helpful, high score = extremely helpful, a = .80). Intervening variables 
were: (4) adequacy of money resources measured by the mean score on seven items from the Family 
Resource Scale related to money (a = .90, 1 = not adequate and 5 = always adequate); (5) frequency of 
use of respite care during the last six months measured on a five-point item as part of the telephone 
interview (1 = not used, 2 = less than once per month, 3 = 1 to 3 times per month, 4 = once a week, 5 = 
more than once a week; (6) parent's perception of the adequacy of relief from childcare measured using 
one item from the Family Resource Scale that ask about the adequacy of baby sitting ( 1 = not adequate, 
5 = always adequate); (7) perception of the adequacy of time resources measured by the mean score on 
nine items from the Family Resource Scale related to time (a= .86, 1 = not adequate and 5 = always 
adequate). The dependent variable was (8) parental perception of stress as measured by the 
Questionnaire on Resources and Stress (QRS-F). Total scale score was used (a = .92) where a low score 
indicated low stress and a high score indicated high stress. Mean scores for these variables are presented 
in Table 4. 
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Results 

Bivariate Analysis 

Use of respite was negatively correlated (-.25) with adequacy of money resources and with the adequacy 
of time resources (-.24). This suggests that families with better financial situations are less likely to 
obtain childcare from public services. The use of respite care was positively correlated (.24) with stress. 
These observations indicate the need to understand the combination effects of other variables with the 
use of respite on perception of stress. 

Path Analysis 

The hypothesized path flow is depicted in Figure 1 and includes the significant bivariate relationships. 
Multiple regression analyses were conducted regressing each variable on all variables preceding it in the 
model. Figure 2 displays the relationships among the variables with significant betas (p < .05). The path 
flow shows the amount of respite received increased as the child's disability became more severe and 
decreased as the adequacy of money resources increased. The child's disability level affected perception 
of the adequacy of time resources directly, with time decreasing and stress increasing as level of 
disability increased. Disability level had an indirect effect on adequacy of time resources through 
adequacy of money resources (Variable 4, indirect effect = -.11), which acted as a moderator variable on 
time resources. The child's disability level also had a direct effect on stress, increasing stress as disability 
level increased, and a strong indirect effect via perception of time resources (Variable 7, indirect effect = 
.19) decreasing the adequacy of time resources. The presence of a spouse only indirectly affected stress 
level, with the strongest path via adequacy of money and time resources. Helpfulness of the social 
support network had virtually no effect on perception of stress. 

The amount of respite care and adequacy of baby sitting has only indirect effects on perception of stress. 
For both variables, this effect was via the effect on time resources. The adequacy of money resources 
was negatively related to respite but positive related to adequacy of baby sitting. This suggests that while 
better financial situations resulted in less publicly supported childcare, it was balance by the ability to 
purchase or find childcare from other sources. The indirect effect of respite care is small (-.03) while the 
indirect effect of adequacy of baby sitting is much stronger (.12). For both variables, positive changes 
result in decreased stress by improving the parent's perception of time resources. 
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Conclusions 

The path analysis demonstrated the complex set of relationships that control the effective impact of 
respite care on parental perception of stress. One of the outcomes expected for respite care is a reduction 
in stress. This analysis suggests that respite care indirectly reduces stress by affecting the availability of 
personal time for the parent. Not surprisingly, the child's level of disability was found to be a dominant 
factor, directly and indirectly moderating the affects of time and money resources and respite care. When 
the relationship between respite care and parental perception of stress was viewed independent of other 
variables, respite care did not appear to have a positive impact. However, when the relationship of 
respite care to stress is viewed with the context of other variables (e.g., the child's level of disability, 
adequacy of monetary resources, and the adequacy of baby sitting), its positive effects on parental 
perception of time resources and through time on stress become evident. 

The role of monetary resources and its interaction with the use of respite care was also evident in the 
path model. This model suggests that public programs can affect the amount of stress experienced by 
parents by providing respite care. However, in the amoimt provided by typical programs, respite care is 
likely to have only a marginal effect on stress. Adding money resources to the family system as through 
cash subsidy program is likely to have a greater impact on stress levels. These analyses suggest that a 
combination of respite care and cash subsidies may be a stronger intervention than either alone. 

The strong role of time adequacy in relationship to parental stress also suggests that interventions with 
families need to assess parent's perceptions of time for themselves and provide supports that increase the 
time and opportunities parents have to care for themselves. Respite care clearly has the potential to do 
this but will need to be matched to parents' personal level of need as well as the child's needs. 
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Tabl6 4 

Moan Scores on Path Analysis Variables 



VamUe 


Mean 


SD 


F^xige 


l.Picesence of spouse 


69.10% 






2. Amount of caie child xequues 


6.97 


3.42 


0tol5 


3. Helpfuln ess of socialnetwoxk 


3.12 


1.09 


1.06 to 9.00 


4. Adequacy of m on ey xesouxc es 


3.35 


1.09 


lto5 


5. Amount of respite caie 


2.49 


1.35 


lto5 


6 . Adequacy of b abysittin g 


2.49 


1.49 


lto5 


7. Adequacy of time xeso uxces 


3.33 


0.91 


1.13 to 5 


8. Pexception of shess 


22.68 


12.59 


0lo49 
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Current Trends in Children's Mental Health 

Central to the efforts to enhance the delivery of broad based services to families of children with severe 
emotional and behavioral problems is the recognition that a new philosophy and set of practice values 
must be a fundamental part of systems change (Behar, 1988; Burchard, et al, 1991; Homonoff & Maltz, 
1991). This new philosophy (e.g., Lourie, et al, 1990; Raiff, 1990; Rormau, 1991; Stroul & Friedman, 
1986; Ventura County Children's Mental Health Services Demonstration Project, 1987) focuses on such 
values as emphasizing family and child strengths, respecting the needs, wants and desires of families and 
children, and maintaining children in the most family-like environment. 

Significant progress has been made in developing child and family services grounded in a family 
strengths based philosophy (e.g., Stroul, 1993), however, to date, programs are only just begirming to 
incorporate the family strengths, family based perspective into program evaluation research design and 
methodology. If we are to maintain our commitment to empowering families from a strengths based 
perspective, and to expand the opportunities for enhanced parent-professional collaboration, we must 
develop irmovative methodologies which consider families as full participants in all aspects of ongoing 
program evaluation research. 

This summary describes one project, undertaken in the State of Maine, to involve parents as researchers 
in collaborative action research. Collaborative action research suggests that each group represented in 
the process "shared in the plarming, implementation, and analysis of the research and that each 
contributes different expertise and a unique perspective" (Oja & Smulynam, 1989; p. 1). Collaborative 
action research represents an attempt to empower people proactively, and to study research problems 
which arise out of a community, rather than solely through the eyes of the researcher. In collaborative 
action research, parents and researchers work together to set common goals and mutually plan the 
research design, collect and analyze data, and report results. Collaboration assumes that researchers and 
parents will communicate frequently and openly throughout the process to avoid possible conflicting 
perceptions and assumptions which result from their different perspectives. 
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The Intensive Child Case Management Program 

This pilot study grew out of an ongoing program evaluation of a new intensive child case management 
program in the state of Maine. The intensive child case management services are part of a continuum of 
child and adolescent mental health services developed and implemented by the state of Maine 
Department of Mental Health and Mental Retardation. This continuum of services 
(e.g. child case management, crisis services, community-based residential services, etc.) was legally 
mandated following agreement of a class action suit between mental health consumer groups and state 
mental health and human services officials, resulting in a substantial decrease in the number of state 
hospital beds for adults, a virtual elimination of state hospital beds for children and adolescents, and 
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increased funding allocations for improving and expanding community based mental health services. Six 
agencies were awarded four year contracts with the state department of mental health to provide 
intensive child case management services. 

We were interested in understanding parents' perceptions regarding their involvement and satisfaction 
with intensive child case management services; therefore, we proposed a collaborative research model in 
which parents were active participants in all aspects of the evaluation of parent satisfaction. 

We want to emphasize several important points about this pilot project: 

• This is a small pilot study&shyp;we were only funded for $5,000. Money is nice, but you don't 
have to have a lot of it! 

• We are trying to accomplish the project in an incremental fashion by using this model in one 
region of the state, and then replicating our efforts in other regions. 

• Above all, we wanted to develop and refine family strengths based research methodology that was 
"family friendly" and could be used by the families themselves. 
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The Pilot Study 

First, we conducted two focus groups with parents of children with serious emotional and behavioral 
disabilities (who received case management services) in order to obtain parents' recommendations 
regarding the conduct of family strengths based research, and to recruit for parent researchers. 

What did we find? 

1 . Parents want evaluation research that doesn't exclusively focus on family dysfunction and 
problems. 

2. Parents feel blamed and judged by professionals already &shyp;they don't want to be evaluated 
from a problem focus. 

3. Parents want researchers to spend more time with them&shyp;i.e., "Come to our houses and spend 
some time with us in order to more accurately observe our family and child situation." 

4. Parents believe that there is no accountability in the system for child outcomes, and they want 
evaluation of service providers to stress accountability and responsiveness to families and 
children. They want an accountable system, which includes ongoing assessment, quality assurance 
and evaluation. 

5. Parents want more collaboration among providers and parents&shyp;and this includes researchers. 
Parents want to participate in the evaluation process through periodic updates, ongoing 
participation in focus groups and participation in data collection. 



Recruitment of Parents as Participants on the Research Team 

We recruited parent-researchers from participants in the focus groups. Due to budget constraints on the 
project, only one parent was able to get paid and work on the project. The parent-researcher kept a 
notebook of her experiences, feelings and thoughts as she participated in this project. 

Creation of Semi-Structured Interview 

Based on thematic analysis of the focus groups' discourse, we developed a semi-structured interview 
aimed at discovering parental satisfaction with intensive child case management services. The interview 
consisted of open-ended questions related to characteristics of the child and family, parental satisfaction 
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with the system of care, service needs and barriers, and history of service use. We spent time assessing 
the interview format in order to insure that the interview was strengths based and family and child 
focused. 

We taped the interviews and transcribed the notes, working in teams for the purpose of conducting 
interviews. The evaluator did the initial interview with the parent-researcher observing. Next, the 
parent-researcher completed two interviews as the lead interviewer, with the evaluator as the observer. 
After each interview, we processed how the interview was conducted, what could be changed and what 
was working well. Four interviews have been completed to date. 

What have we learned so far? 

1. Families find great satisfaction with a case manager who does not take over, causing the family to 
be totally dependent on them, but rather offers suggestions and service options to help the family 
find out what it is that works best for them. 

2. Families are pleased when they do not receive criticism, but instead are given unconditional 
acceptance fi’om their case manager, being applauded for what the family feels are their successes. 



3. Families appreciate and benefit fi’om a case manager who encourages a family to ask questions 
and does not appear to perceive them as a nuisance when they do so. Case managers should give 
families adequate information to insure the best possible communication between them. 

4. Families also appreciate case managers who perceive the family as being comprised of individuals 
who are part of the same whole. Much consideration of other family members' needs must be 
taken into account, strengthening the entire family, if the child is to succeed within her or his own 
home. 

5. Families want and expect a case manager to advocate with them on a consistent basis with the 
schools, their communities, and local and state government, and not remain in the background 
simply hoping that change will magically come about. They want their case managers to be 
knowledgeable about services and to make a strong stand that will directly effect the creation and 
availability of child and adolescent mental health services. 

Return to Top 



Comments from the Parent-Researcher 

Parents as researchers can serve as role models of empowerment for families who are being interviewed. 
It is empowering for families to perceive others parents of children with SED as being a valued part of 
the research process which can directly effect change in the way mental health services will be delivered 
in the future. Parents of children with SED are often perceived as dysfunctional by professionals in the 
system and may be reluctant to confide&shyp;fearing blame for their children's illness. Such is not the 
case when families converse with one another. 

Families who have experienced similar difficulties with the system have little if any difficulty confiding 
their problems to each other. Parents have remarked during the interview that I "must know how they 
feel," and they visibly relax. I feel that a climate of trust directly effects the amount of detailed 
information that can be elicited. Families experience validation when asked to share their "stories" with 
another parent. Parent researchers assume a unique and important role for families, serving to 
authenticate families' experiences through the interview process. 

As a parent researcher who has had the opportunity to interview other parents about their child's case 
management service, I have some recommendations for the training of parent researchers: 
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• Give parent researchers the opportunity to explore their own value conflicts and prejudices before 
they conduct their first interview . I experienced the awful reality of such prejudice during my first 
encounter with a parent subject in her home. Her home did not meet my standards of cleanliness, 
and for a brief moment I wondered if the reason why her children were so ill was because of living 
in such an environment. Fortunately, a light of recognition went off in my head, and I was forced 
to take a long hard look at my own value conflicts. 

• Parent researchers need training to understand the different styles of communication they may 
encounter in interviews. I am a talkative person, and found I needed to slow down for a more quiet 
parent. 

• Parent researchers need the opportunity to practice interview skills. Prospective parent researchers 
should practice interviews with one another. They should focus on tact, diplomacy and speaking 
clearly so families can easily comprehend. Their goals should include developing good listening 
skills to extract the meat from the dialogue, learning to probe for information&shyp;while not 
being overly invasive and interrogatory&shyp;and being careful not to overwhelm the subjects 
being interviewed. 

• Parent researchers need practice learning to focus on the parent subjects, and allowing the home 
environment to provide added information without distracting researchers from the purpose of 
their visit. My first home interview involved kids, phones, dogs and other interruptions. I needed 
to "go with the flow." 

• Give parent researchers many opportunities to discuss their ovm feelings of distress when listening 
to a subject who is expressing similar painful issues that may mirror the interviewers ovm 
experiences. Provide parent researchers with a mechanism to deal with those feelings, both for 
themselves and for the integrity of the interview, so that there is no subtle hint of leading the 
subject into an area which may serve to validate the interviewer, but which could contaminate that 
interview with bias. 
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Introduction 

With completion of field trials for the American Psychiatric Association Diagnostic and Statistical 
Manual, 4th edition (DSM-IV; APA, 1994), we have further refined psychiatric nomenclature. One 
problem, however, is that children and families having difficulties do not always fit into present 
diagnostic schemes. This noncon-formity may be to the detriment of mental health clinicians and the 
children and families they treat. It also may lead to an inability to access services. 

For example, a recent ruling in one state requires a child to have a major psychiatric diagnosis in order 
for child guidance clinic resources to be allocated; the non-specific DSM-IV diagnosis of Family 
Problems- V61. XX is not sufficient criteria for the allocation of services. Clinical experience suggests, 
however, that Family Problems may be the salient presentation for a child and family who are 
considerably distressed and can make significant use of clinical assistance. For many of these families, 
the focus of the clinical effort is toward locating the source of distress. 

The hypothesis tested in this study was that children who present with Family Problems (V61 .xx) are as 
distressed and make use of clinical assistance as much as those with major DSM-IV psychiatric 
diagnoses. 
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Method 

Setting 

This study used data from a chart review of a child guidance center located in a community mental 
health center. The child guidance center was the sole child mental health practice in the community, had 
a sliding fee, and was state funded and completely independent of third party payers. The community in 
which the child guidance center was located was a suburban working class community of 54,000, with a 
median household income of $35,000-40,000. Six percent of the households were below poverty level in 
income, while 2.6% of the households had incomes above $100,000. The racial/ethnic mix of the 
community was 82% Caucasian, 12% African American and 5% of other ethnic origins. 

Measurements 

This study used a retrospective record review of all children seen in a single community mental health 
center. Information abstracted from the records included: 

• Demographic information 

• All diagnoses 

• Dates of all visits 
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• Child global assessment scores, when available 

The diagnosis of Family Problems and five psychiatric diagnoses (Adjustment Disorder, Dysthymia, 
Oppositional Defiant Disorder, Conduct Disorder and Major Depression) 
were examined in detail with regard to: 

• Number of additional diagnoses 

• Follow-up time 

• Number of visits 

• Child global assessment scale scores on presentation 
Return to Top 



Results 

Six hundred thirty eight children were seen more than once between July 1980 and December 1993 in 
this community child guidance clinic. The racial mix of children seen in the clinic matched the racial 
mix of the community. The average age of the children was 10 years old, and these children were seen 
for an average 16 visits (SD=19). The average child global assessment scores for 376 of these children 
was 61 ± 10 (range 20 to 85). 

Family Problems was found to be a presenting problem in 55% of the children who presented, and in 
26% of these children, it was the sole diagnosis. Family Problems (9.6 yrs), adjustment disorder 
(9.8 yrs) and oppositional defiant disorder (9.4 yrs) tended to present at a younger age than dysthymia 
(12.0 yrs), major depression (13.3 yrs) and conduct disorder (12.3 yrs). The average child global 
assessment score for children with a diagnosis of Family Problems was 63, and only children with 
conduct disorder and major depression had CGAS scores that were statistically significantly lower than 
children with family problems. 

Children presenting with Family Problems were followed for as long and for as many visits as most 
other psychiatric diagnoses. On subsequent presentations, these children many times presented with 
more prominent psychopathology. For example, of the 209 children who presented to the clinic for a 
second set of visits, 51 presented with dysthymia, 45 presented with oppositional defiant disorder, 18 
with drug and alcohol problems, 15 with attention deficit disorder, 15 with conduct disorder, and 15 with 
major depression. Five children subsequently presented with either psychosis or bipolar disorder. 
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Conclusion 

Family Problems is a common diagnosis in a child guidance clinic population. Although the inclusion of 
nonspecific diagnoses such as "V" codes may seem as an invitation for children and families with less 
severe disturbances to use services, this study demonstrates that these children may be just as impaired 
as other children with major psychiatric diagnoses. A diagnosis of Family Problems is not predictive of 
less need. Results are also consistent with the hypothesis that Family Problems may be the initial 
presentation of chronic disability in some cases. These tentative conclusions are provocative with respect 
to preventive mental health care and the line between acute psychiatric manifestations and chronic social 
disability. In addition, children presenting with Family Problems tend to be younger, which has 
secondary preventative implications. 

The clinic used in this study was not constrained by pressures that normally require the use of Axis I 
diagnoses in order to allocate resources. It is for this reason that the "V" codes of Family Problems could 
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be used as a sole diagnosis. When clinicians are required to use Axis I diag-noses, as they are in most 
clinical settings, conflicts arise. Any diagnostic nomenclature will have inherent limitations, including 
the difficulties of categorization which lead to the social stigma of labeling. In addition, our present 
nomenclature does poorly in characterizing younger children. Clinicians may be forced to use diagnoses 
that are not appropriate due to these limitations. 
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Introduction 

The prevalence of conduct disorder among children and adolescents is quite disturbing given that 
conduct disorder is often identified as a precursor of more severe emotional and behavioral problems 
(Graham & Rutter, 1973). Empirical research examining causes of conduct disorder typically focus 
either on family fimctioning (Greenberg, Speltz, & DeKlyen, 1994; Patterson, Reid, & Dishion, 1992) or 
peer relationships and school context (Coie & Jacobs, 1993; Coie, Dodge, & Kupersmidt, 1990). From a 
developmental perspective, examining associations between conduct disorder and family relationships is 
critical because these are children's first relationships (Ainsworth, 1989). Theoretically, family 
relationships are expected to be resistant to change because these relationships and interaction patterns 
have been in place longer than ot 
her social relationships (Martin, 1987). 

Given the potential impact of family on conduct disorder, this investigation sought to examine 
associations among family functioning and conduct disorder among children and adolescents currently 
receiving mental health services. Specifically, the present investigation was designed to answer three 
questions. First, how stable are family functioning characteristics and children's conduct disorder over a 
six-month interval? Second, what is the nature of the relationship between family functioning and 
children's conduct disorder? Finally, is the association between family functioning and conduct disorder 
dynamic with changes in family functioning being associated with changes in children's behavior? 
Return to Top 



Methods 

Subjects 

All data included in this investigation were collected as part of the Fort Bragg Evaluation Project 
(FBEP; Bickman, Heflinger, Pion, & Behar, 1992) and included 5 to 17 year old children and their 
parents. Of the 819 participating children, 519 were male. The majority of children participating were 
Caucasian (72%), and resided with two parents (either biological or step-parents: 87%). Twenty-six 
percent were clinically diagnosed as having a conduct disorder (see Table IT 

Measures 

Data collected from the first two assessments of the FBEP were included. Time 1 data were collected 
upon entry into mental health services and time 2 data were collected six months later. While not all 
families were seeking services for conduct disorder, all families were soliciting services for children's 
mental health problems. Parents participated in structured interviews and completed a variety of 
measures at times 1 and 2, some of which included assessments of family functioning and children's 
conduct disorder. 

Family Functioning 



ERIC 

MjMiiffrirndkai 



142 



12/10/98 2:49 PM 



8th. 127 



http://lumpy.finhi.usf.edu/CFSroot/rtc/proceeding8th/8th.127.html 



Parents completed the Family Inventory of Life Events (McCubbin, Patterson, & Wilson, 1981). 
Seventy-one items were dichotomously rated as yes or no by parents. If the event had occurred within 
the past six months, parents responded yes. The family conflict, marital strains, and pile up of stress 
subscales were used in these analyses. Parents also completed the Family Assessment Device (Epstein, 
Baldwin, & Bishop, 1983). Parents rated 60 items on a 4 point Likert scale (strongly agree to strongly 
disagree) in terms of how much it described their family during the previous six months. The 
problem-solving/communication, affective responsiveness, affective involvement, general functioning, 
and behavioral control subscales were included. 

Conduct Disorder 

To assess children's level of conduct disorder, parents completed the Child Assessment Schedule (CAS; 
Hodges, Kline, Stem, C)d:ryn, & McKnew, 1982) and the Child Behavior Checklist (Achenbach, 1991). 
The acting out subscale from the CAS and the delinquency and aggression subscales from the CBC were 
used to index conduct disorder. These three scores were significantly correlated and were standardized 
and summed to create the conduct disorder composite. 
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Result and Implications 

The goals of this investigation were to determine the relative stability of family functioning and 
children's conduct disorder over time, the relationship between levels of family functioning and 
concurrent and subsequent child behavior, and the associations among family functioning and changes in 
children's behavior. Before addressing these questions, the family functioning and conduct disorder 
variables were compared by age and gender. While few gender differences emerged, age was 
significantly associated with all variables, except marital strains, such that having an older child was 
indicative of greater functioning and behavioral problems across both time intervals. 

To determine the relative stability of family functioning and conduct disorder over time. 

Stability was assessed by correlating time 1 and time 2 scores (see Table 2T Family functioning 
showever, these correlations were low enough to indicate that, while time 1 functioning may be a good 
predictor of time 2 functioning, time 1 scores were not a perfect predictor of time 2 functioning. 
Children's conduct disorder scores were somewhat more stable across the 6 month interval, but change 
in conduct disorder was also apparent. 

To determine the relationship between level of family functioning and concurrent and subsequent 
conduct disorder. 

Three hierarchical regression equations were computed to examine the relationships between family 
functioning and children's behavior. In each regression equation children's age and gender were entered 
first and accounted for 1-2% of the variance associated with conduct disorder. Of particular interest was 
the amount of change in R2 upon entry of the family functioning variables (see Figure IT Time 1 family 
functioning resulted in a significant change in R2 when predicting time 1 conduct disorder and 
accounted for 22% of the variance. Similarly, time 2 family functioning variables resulted in a 
significant change in R2 and explained 28% of the variance associated with time 2 conduct disorder. 
Time 1 family functioning variables resulted in a significant change in R2 when predicting time 2 
conduct disorder and explained 1 2% of the variance. 

In sum, family functioning was significantly associated with concurrent levels of conduct disorder across 
both time intervals. Thus, families who are functioning poorly are likely to have children who are 
experiencing behavioral problems. A relationship between levels of family functioning at time 1 and 
conduct disorder at time 2 was also found, however, this lagged relationship was not as strong as 
concurrent associations between family fimctioning and conduct disorder. Since level of family 
functioning at time 1 was not as strongly associated with children's time 2 behavior as family 
functioning at time 2, change in functioning may be occurring. 
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To determine whether family functioning was related to changes in children's behavior from time 1 to 
time 2 . 

In determining the relationship between family functioning and changes in children's behavior, residual 
change scores were used to parallel the multiple regression methodology of the previous analyses. In the 
first step of each equation, children's time 1 conduct disorder scores were entered and accounted for 49% 
of the variance of time 2 conduct disorder. Thus, the remaining variance is variance which is 
independent from conduct disorder at time 1 , or changes in conduct disorder from time 1 to time 2. In 
each of these three equations, age and gender were entered in the second step and resulted in no change 
in R2. Finally, the family functioning variables were entered. As with the previous regression equations, 
the focus was on the amount of change in R2 associated with entry of the family functioning variables in 
predicting change in conduct disorder (see Figure 2). Thus, separate regression equations were computed 
with family functioning at time 1, at time 2, and changes from time 1 to time 2 entered in the third step. 

When predicting changes in children's behavior from time 1 family functioning, no significant change in 
R2 emerged. Time 2 family functioning resulted in a significant change in R2 of .10 when predicting 
changes in children's conduct disorder from time 1 to time 2. To examine the association between 
changes in family functioning and changes in children's conduct disorder, time 1 family functioning 
scores were entered. Since time 1 family functioning scores correlated significantly with time 2 
functioning scores, these commonalities were partialled out. Entry of the time 2 family functioning 
variables resulted in a significant change in R2 of . 1 1 , indicating that changes in family functioning from 
time 1 to 2 were associated with changes in children's behavior from time 1 to time 2. 

These regressions were designed to examine the relationship between family functioning and change in 
children's conduct disorder. The results indicate that level of family functioning at time 1 was not a good 
indicator of change in children's behavior, rather children's behavior changed independently of families 
previous functioning. Time 2 family functioning and changes in family functioning were good indicators 
of change in children's behavior. Thus, children who are receiving treatment, and the families in which 
they reside, seem to be changing together. 

In conclusion, all children in this sample were receiving mental health services, although not necessarily 
for conduct disorder. Across both time periods, family functioning, and to a lesser extent, children's 
behavior, were not stable. Determining causation is difficult. One possible explanation of these results is 
that changes in children's behavior led to changes in family functioning. Alternatively, changes in family 
functioning, for instance, reduction in stress or improved communication skills, may result in 
improvements in children's behavior. From a family systems perspective, both of these hypotheses are 
valid. Systemically, change in one domain should influence change in another. Thus, a model of 
reciprocal causality may be needed, such that changes in children's behavior lead to changes in family 
functioning which causes further changes in behavior. Further research examining this more dynamic 
view of families and children is clearly needed. 
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Figure 1 

Hierarchical Regression Equations 
Predicting Children's Conduct Disorder: 
Change in ^ values after Controlling 
for Age and Gender 
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Figure 2 

Hierarchical Regression Equatbns 
Examining Associatbn 
Between Family Function hg and 
Changes in Conduct D border: 
Descriptions of the Changes in 
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Introduction 

Project Destiny (Designing Educational Support Teams through Interagency Networks for Youth with 
Emotional or Behavioral Disorders) is a three year training and research project funded by the US Office 
of Education and Rehabilitative Services. It is part of an initiative to improve school-based services for 
students with emotional or behavioral disorders (EBD), which was established under the Individuals 
with Disabilities Education Act of 1990 (IDEA). The project assesses the impact of staff development 
and school-wide intervention programs on the performance of students with EBD who are included in 
general education classrooms. We have discussed the entire project elsewhere (Cheney & Barringer, 

1995 : Cheney, Barringer, Upham & Manning, 1995). The present discussion focuses on baseline 
assessments of (a) existing service models, (b) teacher competencies, and (c) student characteristics. 
Return to ton 



Method 

Subjects/Sites. Subjects were 113 middle school students (grades 5-8); and 25 teachers, with an average 
of 1 8 years experience. Subjects were selected from two New Hampshire communities, one suburban 
(pop. < 75,000) and one rural (pop. = 10,000). Using a multiple gating procedure (Walker, Severenson, 
Stiller, Williams, Haring, Shinn & Todis, 1988; Walker & Sevemson, 1992), teachers identified 67 
students with EBD, 6 of whom had been classified as Seriously Emotionally Disturbed (SED) under 
special education law; teachers also identified 46 students whom they considered typical for age and 
grade level. 

Intervention Program. During three consecutive school years, a research team comprised of a special 
educator, a psychologist and a family counselor will provide: (a) monthly didactic presentations to staff, 
(b) bi-monthly case studies , (c) parent support groups, and (d) consultation concerning school-wide 
interventions (e.g., problem solving approaches, social skills curricula, and crisis prevention techniques). 

The framework for these interventions is the Project Destiny Transdisciplinary Model (see Table 1), 
which is based on research in the fields of child and adolescent development (Bowlby, 1988, Kholberg, 
1984), developmental psychopathology (Cicchetti, 1993; Cicchetti & Toth, 1992), and special education 
(Walker, Colvin & Ramsey, 1994). Thiis model program provides guidelines for creating a school 
context that facilitates the social-emotional development of all students, and provides extra support for 
students with EBD. A fundamental assumption of the model is that social/emotional development, a 
primary determinant of school performance, is influenced by four developmental factors, (1) biological 
factors, (2) affective factors, (3) interpersonal/familial factors and (4) cognitive factors 

Measurement Instruments. The research design for the project calls for 15 instruments to be 
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administered at the beginning and end of each school year. This report, however, concerns only the 
following subset of instruments: 

1 . Assessment of School Context Survey (see Table 1), 

2. Teacher Competency Survey (Bratten, 1993), 

3. Social Skills Rating Scale (Gresham & Elliot, 1990), 

4. Teacher Report Form (Achenbach, 1991), 

5. Inventory of Parent & Peer Attachment (Armsden & Greenberg, 1987), 

6. Youth Self Report (Achenbach, 1991), and 

7. Reynolds Adolescent Depression Scale (Reynolds, 1986). 

Using these instruments, informants indicated their agreement/disagreement with the specific aspects of 
the proposed model program; and compared existing service delivery systems to the model. Informants 
also rated the competence of teachers, as well as the behavior, academic performance, social skills and 
emotional functioning of students. Other data (e.g., achievement test scores and teacher's comments) 
were collected using archives, direct observations, and interviews. 
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Results 

Service Delivery Systems. Teachers who completed the Assessment of School Context Survey agreed 
with 23 of the 26 components of the proposed model. Teachers further indicated that the service delivery 
systems in their schools lacked most of the components specified in the model (e.g., family support 
groups, an adequate number of consulting specialists, and adequate planning time for teachers and 
students). All schools used IDEA (1990) as their service delivery model, under which less than 12% of 
students with EBD received services; often these were not the students of greatest concern to teachers. 

Teacher Competence. As reported elsewhere (Cheney & Barringer, 1995), classroom teachers rated 
themselves as having little or no knowledge concerning (a) the course of normal social/emotional 
development, (b) federal and state definitions and regulations pertaining to EBD, (c) the functions of 
community agencies providing services to this population, (d) characteristics of students and their 
families, (e) individual differences among students with EBD, and (f) relevant constructs from the fields 
of mental health and education. As expected, teachers reported greater expertise with respect to 
managing group behavior in the classroom and using appropriate instructional techniques. In general, 
teachers did not feel competent, and reported high levels of fmstration, with respect to integrating 
students with EBD into their general education classrooms. 

Student Characteristics. Using a multiple-gating procedure, teachers accurately differ- entiated youth 
with EBD from typical students; in the sample of 1 13 students, there were 7 false positives and 0 false 
negatives. Students with EBD differed from the typical group on virtually all measures. 

Teachers' reports were also valid indicators of students scores along the intemalizing/extemalizing 
dimension (Achenbach & Edelbrock, 1983). Finally, students who had been classified SED under 
special education law differed significantly from unclassified students with EBD on only three variables: 
the classified group's behavior was rated as more internalizing, and they showed a greater degree of 
idiosyncratic thinking; the unclass-ified students with EBD showed greater elevations on the 
externalizing and aggression subscales. 

A subsequent K-Means Cluster Analysis (see Table 2) yielded three groups of students: The Typical 
group (n = 46) showed well developed social skills and no serious emotional, behavioral or academic 
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problems; these students' self-ratings in all of these areas were generally consistent with those obtained 
from teachers and parents. 

In contrast, the Aggressive/ Academic Problem group (n = 34) scored below average with respect to 
social skills and in all academic subjects. Their TRF scores also showed significant problems with 
aggressive behavior, delinquency, inattention and idiosyncratic thinking. These social-emotional 
difficulties were reported by teachers, but not by the students, who ranked themselves within the average 
range on all YSR subscales. 

Students in the Aggressive/Depressed group (n = 33) were similar to those in the Aggressive/Academic 
Problem group: their TRF scores indicated significant problems in the areas of aggressive behavior, 
delinquency, idiosyncratic thinking and social skill development; unlike the Aggressive/Academic 
Problems group, however, their academic achievement was within normal limits despite indications that 
they were socially withdrawn and well above the clinical cut-off on the depression measure. In addition, 
compared to their teachers' ratings, students in the Aggressive/Depressed group rated themselves as 
having more, and more severe emotional problems. 

There were also significant differences between groups with respect to their attachments to their mothers 
and fathers: as expected, the Typical group demonstrated the strongest attachment to parents and showed 
equivalent attachment to their mothers and fathers; average attachment scores were lower for both the 
Aggressive/Academic Problems group and the Aggressive/Depressed groups. These two groups showed 
similar scores, and both groups appeared to have a stronger attachment to mother than to father. 
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Discussion 

At baseline, all schools relied exclusively on the administrative guidelines in IDEA (1990) as a model 
for service allocation. The majority of students with EBD have not been well-served by this model 
(Carson, Sitlington & Frank, 1995; Chesapeake Institute, 1994; Cheney & Harvey, 1994; Wagner, 
Newman, D'Amico, Jay, Butler-Nalin, Marder & Cox, 1991); our results are consistent with previous 
findings concerning this issue. 

Our baseline data indicated that a small percentage (< 12%) of students with EBD were involved with 
special education programs; in addition, classroom teachers were not prepared to deliver educational 
services to students with EBD. Teachers were aware of students who needed additional support; 
however, they hesitated to make referrals for special education services due to social stigma, parental 
reaction, or problems obtaining timely evaluations and follow-up services. 

Constructing responsive school contexts requires an understanding of differences in students' 
social-emotional development. In the Project Destiny middle schools, students with EBD showed 
important differences with respect to, (a) signs of clinical depression, (b) attachment to father, (c) social 
withdrawal, (d) academic achievement, and (e) self-awareness concerning the types of emotional and 
behavioral problems of concern to their parents and teachers. Such fundamental, developmental 
differences may account for some of the variance in the effectiveness of behavioral techniques 
commonly used with this population (e.g., contracting, contingency management, social skills training); 
we will collect additional data during the next two years of the project, and attempt to further study this 
issue. 

Collaborating with students to facilitate social/emotional development involves structuring their 
interactions with people and materials in the environment according to mutually agreed upon goals. 

In the ideal case, the differential impact of developmental variables on the performance of students will 
determine the kinds of school-based services each student receives; students' needs also will determine 
both the nature of staff development and the types of interactions that will facilitate social/emotional 
development. Determining the concordance of data concerning students' needs collected from families, 
teachers, community specialists and, most importantly, the students themselves, can provide a basis for 
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collaboration; for example, compared to students in the Aggressive/Academic Problems group, students 
in our Aggressive/Depressed group seemed to be more aware of the problems identified by their 
teachers, and therefore may be relatively more amenable to collaborating on solutions. 

Focusing on how an individual interacts with people and materials in the environment is a common 
practice in clinical settings; however, this notion may be new, and seem impractical to middle-school 
teachers and administrators, who have been mainly concerned with large groups of students, rather than 
individuals within the group. Meaningful inclusion of young teens with EBD in general education 
requires that teachers, administrators, parents and students share an understanding of how to best 
facilitate the individual student's social emotional development. Presently, the absence of a shared 
developmental model is an obstacle to this sort of collaboration , and thus blocks the full inclusion of 
students with EBD in the educational process. Over the next two years. Project Destiny will assess the 
utility of an empirically based developmental model for inclusion of students with EBD in general 
education. 
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Introduction 

Specific special education initiatives in Illinois have been developed to implement wraparound models 
similar to those found in mental health and child welfare as a way to improve outcomes for children with 
emotional and behavioral disabilities (EBD) and their families. These programs provide and coordinate 
services that address all life domains, are strength-based and family-centered. Base-line data were 
collected on 81 students which included descriptive characteristics, student and family outcomes. 
Improvements were seen in family functioning, emotional and behavioral functioning of the students, 
and reduction in the use of psychiatric hospital days. 

During the 1993-94 school year, the Illinois State Board of Education (ISBE) funded the Emotional and 
Behavioral Disability Partnership Initiatives. These projects have been involved in developing 
individualized services and supports for students with EBD in natural home, school and community 
settings. Given that these students have complex needs and often require multiple services, a critical 
component of these projects was to develop interagency networks that facilitate coordinated and 
comprehensive plans for students and families. These initiatives cultivated partnerships among 
community agencies and encouraged new ways to use existing resources and improve service delivery 
for students with emotional and behavioral disabilities (EBD). Along with the commitment to fund the 
operation of the projects, ISBE funded an evaluation of the projects. 
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Methodology 

The evaluation of the EBD Partnership Initiatives is a longitudinal study of the effects of the individual 
supports and services on student, family and system outcomes. Various instruments were used to collect 
information on demographic characteristics, family functioning, parent satisfaction, student emotional 
and behavioral functioning, academic performance and school placements, out-of-home care and 
hospitalizations, and service receipt. For the purpose of this paper, five research questions were 
addressed. 

1. Did family functioning improve during the course of project participation? 

2. What factors were associated with parent satisfaction? 

3. Did student functioning improve during the course of project participation? 

4. Did out-of-home care change with respect to the number of days and level of restrictiveness? 

5. How were local educational systems affected by the wraparound initiatives and what was their 
impact on local systems of care? 



Sample Characteristics 

Base-line data were collected on 81 students who were referred for services from September, 1993 
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through June, 1994. Students were predominately male (82%) with an average age of 14.64 years, 
ranging from 7.9 to 19.3 years. At the time of referral, 59% of the students scored in the clinical range 
on Internalizing Behavior and 50% scored in the clinical range on Externalizing Behavior as measured 
by the Children Behavior Checklist (CBCL; Achenbach, 1991). Forty-one of the students (51%) had at 
least one out-of-home placement prior to project participation. 

With respect to family characteristics, slightly more than half (55%) of the families were two parent 
households. The majority of families were Caucasian (85%), followed by nine families who were 
African-American (1 1%). The average family income was $26,853.00. 
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Results 

The first year evaluation results of school-based wraparound initiatives in Illinois demonstrated positive 
outcomes; however, these outcomes should be viewed with caution for two reasons. First, they represent 
findings from the first year of program participation and as yet sustained results have not been measured. 
Second, Time 2 data was not completed for every student. Preliminary analyses were conducted to test 
for selection bias. No differences were found between students with Time 2 data vs. students without 
Time 2 data. The tables in this summary reflect the sample number for each instrument. 

Family Functioning 

Family functioning was measured using the Family Adaptability and Cohesiveness Evaluation Scales 
(FACES: III; Olson, Portner & Lavee, 1983). Families completed these instruments at the time of 
referral and approximately one year after services began (see Table U. Families who received 
wraparound services showed a significant improvement in adaptability and cohesiveness. 

Family Satisfaction. 

Family satisfaction was measured using nine items on the Time 2 Parent Survey. Families were asked to 
rank items addressing such issues as how the project improved family life, increased their ability to care 
for their child, and getting to know other parents. An overall satisfaction score was summed across the 
nine items. The results indicate that satisfaction with the project was associated with parent(s)' 
perception of inclusion in the decision making process regardless of the specific services that they 
received. With respect to new services, a positive correlation (r = .44*) was found with family 
satisfaction and assistance from project staff in obtaining government benefits (e.g., AFDC, SSI, Food 
Stamps). 

Student Functioning 

Emotional and behavioral functioning was measured using three instruments: the Children Behavior 
Checklist (CBCL), the Child and Adolescent Functioning Assessment Scales (CAFAS; Hodges, 1990), 
and the Teacher Report Form (TRF; Achenbach, 1991). All three instruments were completed at the 
time of referral and approximately 1 year later. No significant change was found from the TRF. Table 2 
gives the results from paired t-tests for males and females from the CBCL. The results indicate that the 
females students showed significant improvements in Internalizing Behavior. Specifically, 
improvements were seen in the Withdrawn and Attention Sub-scales of the CBCL. Male students 
showed significant improvements on the Social Problem and Thought Problem sub-scales. Table 3 g ives 
the results of paired t-tests for the sub-scales of the CAFAS. The results indicate that students who 
received individual supports and services through the Emotional and Behavioral Disability Partnership 
Initiatives improved in Role Performance and Moods as measured by CAFAS. 

Out-of-Home Care 

Out-of-home care was monitored using the Restrictiveness of Living Environment Scales (ROLES; 
Hawkins, Almeida, Fabry & Reitz, 1992). The instrument was completed at time of referral and 1 year 
after participation in the project (see Table 3). Students receiving wraparound plans had fewer 
out-of-home placements and spent fewer days in psychiatric hospitals after receiving wraparound 
services. 



O 

ERIC 



158 



12/11/98 8:46 AM 



8th.l43.html 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.143.html 



Impact on Local Educational Systems and Service Systems 

A self-evaluation tool was developed for project staff to examine their progress relative to specific target 
goals. These goals adhere to the basic system of care principles. It is believed that the way services are 
delivered, and the organization of services within the services system, are begirming to be more 
responsive to the individual needs of students with EBD and their families. Listed below are strategies 
developed and implemented by project staff to further their system change outcomes. 

• Increase parental input in decision making role. Work with state level Illinois Federation of 
Families to train parents as facilitators. Develop forums (group meetings/support groups) where 
parents can exchange information and gain advocacy skills. 

• Merge project staff positions with special education districts or community agencies to continue to 
facilitate the wraparound approach (i.e., school social worker&shyp;family service facilitator). 

• 

Merge project interagency councils with the Local Area Networks (LANs). (LANs is a mental 
health reform in Illinois bringing individualized care services to children with EBD and their 
families to their communities.) 
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Discussion 

Historically, the education system has been more reluctant than mental health and child welfare systems 
to incorporate the wraparound tenets into their daily practice. This reluctance clusters around three main 
concerns. At the school level, is the teachers' receptivity to implement strategies and interventions 
needed to assist students in more inclusive settings (Clarke, Schaefer, Burchard & Welkowitz, 1992). At 
the administrative level, is the concern about liability and fiscal responsibility for providing the full 
array of life domain services identified in wraparound plans. And, at the community level, is the concern 
about the tension among interagency teams and who should coordinate and implement the plans (Lourie, 
1994). These issues create challenges for the comprehensive and sometimes nontraditional aspects of 
wraparound in the educational system. 

The findings from the evaluation of the Illinois Emotional and Behavioral Disability Partnership 
Initiatives begin to demonstrate how the wraparound process can be an effective means to serve students 
in school environments. The education system is a natural point of entry for children with EBD and they 
are in a position to develop partnerships with families and community stakeholders. Improvements in 
family and student outcomes, as well as the reduced need for psychiatric hospitalizations were 
demonstrated. Finally, project staff believe that the implementation of this project and approach has 
changed the larger system of service delivery in these schools and this impact may last beyond any 
artificial boundaries of the life of a project. 
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Introduction 

The rate at which schools identify children as having serious emotional disturbances (SED) for the 
purpose of special education has been the subject of considerable controversy in recent years. The 
national rate (.89% of enrollment) represents roughly one-third to one-half of the estimated prevalence 
of children with emotional problems who need special education services (Institute of Medicine, 1989; 
U.S. Department of Education, 1994). This rate has remained quite stable since the national child count 
data was first collected and has been widely interpreted to mean that, in general, children with SED are 
significantly underidentified. 

The national figure, however, does not tell the whole story. There is substantial variation across states in 
the rate of identification, ranging from .04 % to 2.15% of enrollment for the 1991-92 school year. This 
50-fold difference between the lowest and highest rates suggests that states are identifying and serving 
children with emotional disturbances very differently. While the highest rate states fall within the 
estimated prevalence range (2-3%), most states fall well below these figures (U.S. Department of 
Education, 1994). 

The present study examines relationships among state SED identification rates and a variety of economic 
and demographic variables. The purpose of these analyses was to investigate variables which may be 
relevant in predicting, planning for, and influencing state identification rates for children with SED. At a 
broader level, we examine the public policy implications of these relationships and describe further 
investigation which will clarify the picture. 
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Method 

State identification rates for children with SED for 1 992 were calculated from the national dataset used 
to produce the annual report to Congress. This dataset includes the number of children identified with 
SED in the public school system in each of the 50 states and the District of Columbia, as well as 
estimated enrollment figures for each state. The identification rates that were used are identical to those 
published in the Fifteenth Annual Report, within the limits of rounding error, except for isolated cases in 
which the figures were updated after the Annual Report went to print (U.S. Department of Education, 
1993). For the present analyses, we used the ratio of the SED child count to the estimated enrollment for 
each state (SEDRATE). 

Economic and demographic variables were extracted from the National Center for Educational Statistics, 
summarizing information relevant to education. On the basis of previous related work, we selected a set 
of economic and demographic variables judged likely to relate to the identification of SED children. 
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State demographic variables included geographic region, using NCES regional divisions (REGION); 
state per capita income (PERCAP); state per pupil revenue (PERPUP); and a ratio of per pupil revenue 
to per capita income (INDEX). According to the Education Digest, INDEX "reflects monies raised to 
educate the average public school student relative to the taxpayer's ability to pay" (National Center for 
Education Statistics, 1992). The primary race variable (WHITE) represented the percent of Caucasian 
students enrolled in public elementary and secondary schools in 1990. It should be emphasized that this 
distribution was for the public school population as a whole, not the ethnic distribution of the special 
education population, wherein minorities are often overrepresented. Finally, as a relative measure of 
school achievement level in the state, eighth grade math proficiency scores (MATHACH) were included 
in the variable set. 

State economic variables included in the analyses were: (a) Total Expenditures Per Capita (TEPC), 
which consisted of state and local government expenditures for education services, social services and 
income maintenance, transportation, public safety, environment and housing, governmental 
administration, interest on general debt, and other general expenditures, including intergovernmental 
expenditure to the Federal Government; (b) Education Expenditure Per Capital (EEPC), which reflected 
state and local government expenditure on all education ; and (c) Elementary and Secondary Education 
Expenditure Per Capita (ESEEPC), representing state and local government expenditure on elementary 
and secondary education. 

Elementary and Secondary Education Expenditure Per Capita figures, per pupil revenue, and per capita 
income figures were ranked and then converted to quartiles for the purposes of the ANOVAs reported 
below. In each case, the first quartile represents the states with the lowest incomes or expenditures while 
the fourth quartile represents the states with the highest. Regional divisions, expenditure per capita 
quartiles (EXPQRT), per pupil revenue quartiles (PRPQRT), and per capita income quartiles (PCIQRT) 
were compared in separate ANOVAs to examine the effects of these variables on SEDRATE. 
Correlation analyses were conducted to explore other relationships among the variables. A stepwise 
regression was performed, including all variables which displayed moderate relationships (r > .30) with 
SEDRATE. Finally, the earlier ANOVAs were run again, using as covariates those variables which 
entered into the stepwise regression to determine which variable possessed the best predictive value. 
Return to Top 



Results 

SEDRATE was found to differ significantly across regions (F = 6.47; p = .0009). A 
Student-Newman-Keuls (SNK) means comparison revealed that Northeast states have significantly 
higher rates of SED identification than Midwestern, Southern, or Western states. The latter three do not 
differ from one another. 



SEDRATE also differed across expenditure quartiles (F = 4.09; p = .01). The SNK test indicated that 
states spending the least on elementary and secondary education per capita (EXPQRT =1) identify 
significantly fewer SED children. Quartile 2,3, and 4 did not differ on SEDRATE. 

Per pupil revenue quartiles differed significantly in SEDRATE (F = 5.56; p = .0024). SNK means 
comparisons revealed that states with the highest per pupil revenue (PERPUP = 4) had higher rates of 
SED identification than states in quartiles one and two. States in the third quartile also had higher rates 
than states in quartile one. 

The per capita income quartiles ANOVA also yielded a marginally significant effect (F = 2.95; p = .04). 
The SNK means comparison did not reveal significant differences among the means, although there was 
an orderly trend from the lowest per capita income states (PCIQRT = 1; SEDRATE = .57) to the highest 
(PCIQRT = 4; SEDRATE = 1 .05). 



The correlation analysis revealed three variables that were moderately related and positively related to 
SEDRATE: per pupil educational revenue (PERPUP; r = .49), per capita income (PERCAP; r = .46), 
and the NCES Index (INDEX; r = .31). No other significant correlations were found. 

The stepwise regression procedure yielded a single variable in the final equation (PERPUP). The model 
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R-square was . 24 , indicating that about one-fourth of the variation in SED identification rate can 
accounted for by variation in the amount of educational revenue per pupil in the state. 

The REGION ANCOVA with per pupil revenue as a covariate yielded a marginally significant regional 
effect (F = 2.53; p = .07). Comparison of adjusted least squares means indicated that Northeastern states 
continue to show higher identification rates than Southern and Western states; no other comparisons 
were significant. 

The EXPQRT ANCOVA with per pupil revenue as a covariate also yielded a marginally significant 
effect (F = 2.56; p = .07). Comparison of adjusted least squares means revealed that the states in the first 
quartile (i.e., lowest expenditures) identify significantly fewer SED students than states in the second 
quartile (p = .02) and states in the fourth quartile (i.e., highest expenditures) identify marginally 
significantly fewer SED students than states in the second quartile (p = .06). Other comparisons were not 
significant. 

The PCIQRT ANCOVA with per pupil revenue as a covariate did not yield a significant per capita 
income quartile effect. 
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Discussion 

Examination of the correlates of states' SED identification rates reveals that demographic and economic 
differences across the country may play role in the identification process. While these relationships do 
not explain the variation in SEDRATE, they point to some factors that may play a role in that variation. 

While per pupil revenue is the strongest (and the only significant) single predictor of SED identification 
rates, it accounts for relatively little of the state variation. Further, when the effects of per pupil revenue 
variation are removed, states continue to show marginally significant effects for region and expenditure 
on elementary and secondary education. 

While the significant effects of economic variables suggest that educational revenue is important, there 
are clearly other variables not included in the present analysis which also contribute to the range in SED 
identification rates. The continuing regional differences may indicate the need to examine social and 
political characteristics of states in order to better understand that variation. 

It is perhaps equally illuminating to consider those characteristics of states which did not relate to the 
target variable. Size of the state, racial makeup (percent White), expenditures on human services, and 
achievement measures were all unrelated to the rate of identification of students with SED. These 
features which are sometimes thought to be associated with prevalence of SED are either unrelated or 
their relationship is masked by other contravening effects. 

In sum, the present analyses offer some indications of the source of the variation in identification of SED 
students and hint at the significance of other constructs not included in the study. Further investigation, 
following the directions suggested above, is warranted to better understand how states identify children 
with emotional disturbance. 
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Introduction 

This paper used administrative data to compare the characteristics of children and adolescents with SED 
from an urban county who were hospitalized in a regional state mental hospital with those of youth 
placed in special educational settings. This study also examined the array of public sector mental health 
services provided to these youths prior to their hospitalization or placement. The questions surrounding 
this investigation included: 

• How likely is it for children who were hospitalized for SED to have received services in the 
special educational system during a previous school year? 

• Are children who are treated in a state psychiatric hospital more likely to receive ambulatory 
treatment than children who receive special educational interventions for SED? 

• What demographic and risk factors may differentiate between children and adolescents served in 
the two service systems? 
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Method 



Data Sources 

Data for this study came from large client-specific data sets for the 1991-1992 fiscal year. Mental health 
services information was extracted from Medicaid paid claims for inpatient, outpatient, and partial 
hospitalization for all eligible beneficiaries in a large northeastern city; from the county reporting system 
for recipients of specialty mental health services in the public sector; and from the regional state mental 
hospital (SMH). Each of these databases was limited to the 1992 fiscal year and to county residents. 
There were 10,942 MA records, 8391 county records, and 101 state mental hospital records. 

The State Bureau of Education provided statewide data on children identified as having a serious 
emotional disturbance as prescribed in P.L. 94-142 and its amendments, or children placed in emotional 
support classrooms. These records, which were entered directly on site when Individual Educational 
Plans (lEPs) were submitted for the annual head count, contained demographic and placement 
information, including services received, location of intervention, percentage of time in special setting, 
and suspensions from school. In 1992 there were 24,437 records for the entire state, and 3433 for the 
county under study. 



Analyses 

Each of the Medicaid-reimbursed claims was identified by a sixteen-digit recipient code, assigned at the 
time of first request for approval of service. These numbers were issued sequentially by the Office of 
Medical Assistance. County mental health records carried a unique nine-digit client-specific eligibility 
number. In the state mental hospital records, eight-character facility-specific case numbers were assigned 
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to each patient, and social security numbers were also available on some client files. Special education 
(BSE) records contained an identification number constructed of the nine-digit administrative unit 
number which represented the school where the child's lEP was implemented, and up to nine more digits 
identifying the individual learner. However, except for the county mental health record numbers, which 
were less subject to change, none of these identifiers was unique and truly client-specific. Movement 
from one SMH inpatient facility to another or from one BSE setting to another, led to the attribution of a 
new and unrelated case number. Consequently, any individual receiving multiple services in the public 
sector could have been identified with several codes within and between service provider agencies. 

In order to merge the data and determine an unduplicated count of children who received multiple 
services, and what array of services they received, all the while protecting confidentiality, an identifier 
was constructed of patient initials, birthdate and sex, using an algorithm comparable to those described 
elsewhere (Donaldson & Lohr, 1994; Goerge, Van Voorhis, & Lee, 1994). Successive sorting and 
merging of the data yielded a pool of 26,954 individuals. This pool represented about 8.7% of the county 
population for this age group (uncorrected 1990 U.S. Census count for children aged 3-17 in Urban 
County = 309,712). Descriptive analyses of socio-demographic and risk factors for the hospitalized and 
school-identified children were performed. Odds ratios and their confidence intervals were calculated to 
compare the special education and psychiatric inpatient service systems (Kahn & Sempos, 1989). 
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Results 



Descriptive Analysis 

There were 797 children and adolescents who received inpatient psychiatric care in the county, and 
3,433 who had an lEP during the 1992 fiscal year. There was a majority of male youth in both inpatient 
mental health and special education, but the ratio of males to females in special education (4:1) was 
much higher. The mean age of children receiving special educational services was 12.7 years (SD = 2.9); 
mean age of inpatients was 12.8 years (SD = 3.1), and those in the regional state mental hospital was 
14.3 years (SD = 2.1). Special education and community-based mental health services served a 
proportion of minority clients which was consistent with local demographics, but inpatient facilities 
served a much lower proportion of minority clients. 

Of the 797 inpatient psychiatric clients, 101 were hospitalized in the regional SMH for children and 
adolescents. Twenty-nine percent of these patients had been hospitalized in a community psychiatric unit 
prior to transfer to the state hospital, for lengths of stay ranging from one day to more than six weeks; 
more than half of these prior hospitalizations had been billed as emergency interventions. More than half 
SMH youth (60%) had received community-based outpatient services in 1992, and 5% had also been 
served in partial hospitalization. Fifteen percent received intensive case management and four patients 
had also received "wraparound" mental health services in their homes or communities. Only four 
patients had been identified as SED in the schools prior to their inpatient stay. The majority of these 
patients (70%) were Medicaid eligible. Except for gender (100% male) and race (84% African American 
or Latino), the 26 youth who were admitted to the state facility from correctional facilities were 
comparable to the hospital population (28% prior hospitalization, 
of which 60% were emergency). 



Of the 3,332 youth with active lEPs who were not in the SMH, one third received mental health services 
in 1992. Of these 3% had an inpatient psychiatric stay in the community (85% of which were billed as 
emergency admissions), 17% received outpatient treatment (mean = 76 units, SD =135 units), and 4% 
also had partial hospital treatment (mean = 20 units, SD = 47 units). One out of five of these youth lived 
in a state or private institution and 3% lived in correctional facilities. About two-thirds of children and 
adolescents with SED spent more than 60% of their day in a BSE setting, and were therefore separated 
from their agemates; 25% were not in their home school and were therefore separated from their 
neighbors. Eighty-two percent of those who spent more than half their school day in a separate 
placement were male, as were 86% of those who were suspended at least once; 80% of those who were 
suspended were African American or Latino youth. Nearly half (44%) of those in special education in 
1992 had been referred for evaluation prior to 1985, and 5% had received inpatient mental services in 



O 

me 



169 



12/11/98 8:48 AM 



8th.l53.html 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.153.html 



community-based facilities prior to 1992. 

Odds ratios (OR) and their 95% confidence intervals (Cl) were calculated to test for reliable differences 
between youth served in these two service systems. Of those who received mental health services in the 
community in 1992, children who were hospitalized in the regional SMH were almost six times as likely 
as those in BSE to have received outpatient services in 1992 (OR = 5.9 ; Cl 4.1, 8.6). Three percent of 
SMH patients and one percent of children receiving BSE services also received inpatient services in FY 
1992, and there was no difference in the rates of partial hospitalization (about 4%) between the two 
groups. African American and Latino youth were more likely than Euro-American youth to receive only 
BSE services (OR = 1 .6, Cl 1 .4, 1 .9); Euro-American children were twice as likely as African American 
and Latino youth to receive BSE services in an institutional setting (OR = 2.2; Cl 1 .8, 2.6), and about 
twice as likely to receive community-based inpatient psychiatric services (OR = 2.0; Cl 1.6, 2.4). 

Euro- American youth were marginally more likely to receive mental health treatment in the SMH (OR = 
1.5; Cl 1, 2.3), but African American and Latino youth were more likely to be in the forensic unit of the 
SMH (OR = 3.5; Cl = 1,11). 
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Summary and Conclusions 

Integration of data sets from several service systems for children with serious emotional disturbance 
served in the public sector of an urban county yielded records for about 8.7% of the child and adolescent 
population. The current study sought to identify differences in demographic patterns and levels of care 
within and between state psychiatric hospital and special education service systems. Descriptive analysis 
of recipients of these two services for children revealed few system-specific differences: both 
represented greater than 70% male and 70% or more African American and Latino youth; 70% of the 
state hospital population was Medicaid eligible. State hospital patients tended to be slightly older than 
children served in special education; youth in forensic placements in the SMH were older. Youth in 
more restrictive placements in both service systems were disproportionately male (82% to 100%) and 
African American or Latino (80% to 84%). 

The greatest contrast between the systems was the proportion of children who received other mental 
health services during the 1992 fiscal year: 64% of those in the SMH received other mental health 
services (particularly outpatient treatment) during the year, compared to about one-third of youth in SED 
placement in the BSE system. It was quite unlikely for children hospitalized in the state psychiatry 
facility to have been placed in SED special education services prior to hospitalization; this suggests that 
these patients were functioning adequately in the school setting, that their behaviors were not disruptive 
in spite of their serious emotional disturbance, or that their emotional and psychiatric needs otherwise 
went unnoticed in the classroom. This issue deserves further research. 

Developing integrated service system data sets like the one described here and in Goerge, Van Voorhis 
and Lee (1994) is a labor-intensive undertaking, but one that produces an invaluable resource for 
program planners and service researchers. Analysis of the unduplicated count of recipients across service 
systems can assist in identifying trends in service delivery, patterns of cross-system communication, and 
dimensions of risk factors and resilience in the community. 

Return to Top 



References 

Donaldson, M. S., & Lohr, K. (Eds.). (1994). Health data in the information age: Use, disclosure, and 
privacy. Washington, DC.: National Academy Press. 

Goerge, R., Van Voorhis, J., & Lee, B. J. (1994). Illinois' longitudinal and relational child and family 
research database. Social Science Computer Review, 12(3), 351-365. 

Kahn, H. A., & Sempos, C. T. (1989). Statistical methods in epidemiology. New York: 




17P 



12/11/98 8:48 AM 



8th.l53.html 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th. 1 53.html 



Oxford Press. 
Return to Top 



Suzanne Markel-Fox, Ph.D., NCSP 

NASMHPD Research Fellow 

E-Mail: suzanne@server.section.upenn.edu 

Edna Kamis-Gould, Ph.D. 

Research Assistant Professor 
E-Mail: edna@server.section.upenn.edu 

Kathleen A. Foley, M.H.S.A. 

Project Manager 

E-Mail: foley@server.section.upenn.edu 

Center for Mental Health Policy and Services Research 

University of Pennsylvania Medical School 

3600 Market Street, 7th Floor 

Philadelphia, PA 19104-2648 

215/ 662-2886 Fax: 215/349-8715 

Return to Top 







12/11/98 8:48 AM 



8th.l57.html 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th. 1 57.html 



I Home I I search 1 1 index I 

8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

One Year Outcome Findings of the Vanderbilt School-Based 
Counseling Evaluation Project 

Authors 



Introduction Method Results Discussion References 
Return to Table of Contents 



Introduction 

The Vanderbilt University School-Based Counseling Evaluation Project (SBCEP) is a federally funded, 
children's mental health services research program designed to evaluate the effectiveness of a 
collaborative in-school counseling program relative to a tutoring program and traditional community 
based services. Our program addresses four primary areas: (1) the early identification of disturbance in 
at-risk youngsters; (2) the relative accessibility and utilization of psychological services; (3) the 
prevention of costly and restrictive placements; and (4) the effects on students' adjustment. 

Estimates of psychological and behavioral disturbance among children from socially and economically 
disadvantaged backgrounds are overwhelming. Unfortunately, these at-risk youngsters often do not 
receive needed psychological resources, whether it be due to a lack of appropriate and accessible mental 
health services, or because their families lack the transportation or finances necessary to obtain 
treatment. 

This evaluation project was grounded in pilot work demonstrating the feasibility of executing a 
collaborative system of care between mental health and school personnel in the school setting. In our 
school-based programs, qualified, trained and licensed mental health clinicians provided a range of 
services to youngsters at school. The Vanderbilt school-based counseling programs first were 
established in three pilot schools and provided comprehensive mental health services to youngsters 
referred by their classroom teachers. Teacher, principal and parent anecdotal report, together with 
clinical observation, suggested the programs were successful. In light of the recent political and 
economic climate of decreasing availability of funds for non-traditional mental health services, a 
systematic evaluation of the effectiveness of the program was proposed. The SBCEP represented an 
experimentally controlled effort to demonstrate the impact of school-based services on service use, 
accessibility, and children's mental health. 
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Method 

Nine Metropolitan Nashville schools were selected and invited to participate in the research. Three 
schools served as comparison sites, with 30 - 40 yoxmgsters from each school referred for traditional, 
community-based, out-patient treatment; and six were treatment sites, with 30 - 40 children from each 
school randomly assigned to receive either school-based counseling (SBC) or academic tutoring (AT). 
These groups allowed us to experimentally determine the effectiveness of our school-based counseling 
services. 

In all schools, yoxmgsters were selected for participation based on a set of multi-informant indicators of 
psychological adjustment. Based on research suggesting that child self-report and teacher referrals may 
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overemphasize certain problems and underemphasize others, student selection was based on school-wide 
mental health screenings conducted with children, their peers, and teachers. 

These assessments consisted of a series of mental health measures and were administered twice yearly. 
Specifically, measures were selected/designed to tap six domains of psychopathology, including: 
delinquency, aggression, hyperactivity, depression, anxiety, and somatization. Reports from each 
informant were weighted based on the relative validity of specific informants' reports when considering 
a particular area of psychopathology. What results from this combination of informants and weighting of 
scores is a total psychopathology score which allows us to select children representing a broad range of 
emotional and behavioral disturbances. Concurrence among at least two reporters was a necessary 
condition for peirticipant selection. Therefore, if a youngster self reported very high levels of depression 
but the report is not confirmed by teacher or peer report, that child was not selected. A score of at least 
one standard deviation above the mean was required in order to be considered eligible for enrollment. 

In addition to a total psychopathology score, data from these school-wide screenings were used to 
determine a particular child's primary problem area, the domain of psychopathology where difficulties 
were most severe. A number of our analyses were centered around this primary problem domain because 
we hypothesized that improvement in functioning would be most evident when a child's most serious 
emotional or behavioral problem was considered. Also, for each child identified, we determined a 
secondary problem area which represented scores on the domain of psychopathology for which they 
received the second highest score. 

Our program also included a family interview prior to enrollment. Therefore, for each child, we had 
teacher, self, peer and parent report of adjustment over time. For the most pent, we considered the 
weighted sum of the school informants together, given their similar context, and the parent report as a 
separate score. 

Students were in the 2nd to 5th grades at the time of our initial assessment (Spring 1993). They ranged 
in age from 7 to 14, with a mean age of about 10 years, and about 63 % were from minority 
backgrounds. Greater than 70% of the students enrolled in these urban schools received free school 
lunches, an indication of the level of poverty. Over 20% of students in these public schools evidenced 
serious behavioral and emotional problems. Based on our home interviews, it was evident that the 
families and children we treated would not be receiving consistent mental health services if not for our 
program. 

Counseling services were provided at school, during regular school hours. They were comprehensive, 
and could include combinations of individual, group, parent treatment, family therapy, teacher 
consultation, and psychiatric consultation&shyp;in fact, any services traditionally found at local 
community mental health centers. Some of the strengths of having a school-based counselor included: 

• accessibility for consultation with teachers regarding classroom management; opportunities for 
immediate feedback regarding behavior management strategies; 

• readily accessible observation of youngsters in structured and unstructured settings; 

• more realistic understanding of child's academic environment; 

• ability to educate teachers regarding the kind of information that is most useful in developing a 
treatment plan; 

• immediate accessibility; 

• school as a safe place fostering parent/family involvement in treatment; 

• availability for participation in school team meetings; 

• ability to provide teachers with a different perspective on students; 
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• early identification of student difficulties, ability to teach school personnel to recognize early 
indicators of maladjustment; 

• more consistent treatment participation; and 

• ability to support school-wide curriculum for school-wide impact (e.g., anti-drug use programs, 
anti-violence programs). 

In regards to psychopathology, our outcome measures could be placed into one of four groups: 

1 . Primary Problem Domain, which, as mentioned earlier, represents the problem area for which a 
child received the highest rating prior to services. Thus, for one child this variable might represent 
a standardized depression score, whereas for another child it might represent an aggression score; 

2. Total Problem Score, which represents the sum of the various measures, across all domains; 

3. Internalizing Problems, which include depression, anxiety, and somatization; and 

4. Externalizing Problems, which represent aggression, delinquency, and hyperactivity. 

These outcome measures also were grouped by informant, including: Teacher, Self-Report, Peer, and 
Parent. 

These were our dependent variables. All analyses compared the functioning of our Counseling versus 
Tutoring children at the end of the first academic year of services, adjusting for their functioning at the 
beginning of the year. The primary independent variable was Group, which was the Counseling versus 
Tutoring comparison. However, we also were interested in the relation between outcome and several 
other factors: We were interested in whether treatment was more effective: (1) with girls versus boys; (2) 
with older versus younger children; and (3) with children whose primary problem area was Internalizing 
Problems versus children whose primary problem area was Externalizing Problems. So we tested the 
interaction of each of these effects with Group measures. 
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Results 



It was the Primary Problem Area which we believed would be the most likely to change as a result of 
our intervention. When parent report of a child's primary problem area was considered as the outcome 
criterion, our data revealed no significant differences between AT and SBC children. In fact, when we 
controlled for the level of disturbance prior to our intervention, a trend emerged for students enrolled in 
SBC to do marginally worse than those youngsters enrolled in AT. This finding is consistent with 
previous research from Weisz and Weiss who found that 6 months following service provision, parents 
of children in the control group reported no significant improvement over children in the treatment 
group. 

Similarly, a trend was evident in parents' reports of their children's social strengths: Although 
nonsignificant, parents' report of students enrolled in AT indicated marginally greater social competence 
than students receiving SBC at the end of one year of treatment. When parents' reports on the CBCL 
TOTAL scores were considered, the trend for greater improvement in ftmctioning again favored AT 
children over SBC children. When scores for internalizing and externalizing difficulties were analyzed 
separately, however, this trend disappeared. 



When we looked more closely at children's primary problem areas, grouping them according to whether 
they represented externalizing behavioral problems or internalizing difficulties, a significant Group x 
Primary Problem Area interaction emerged. Specifically, for children with internalizing problems (i.e., 
depression, anxiety, somatic complaints), enrollment in SBC resulted in more gains in parent reported 
social skills, whereas for children with externalizing problems (i.e., delinquency, hyperactivity, 
aggression), the opposite was true; students enrolled in AT scored better on measures of social skills. 
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The combined, total psychopathology scores from teacher, self and peer ratings revealed no significant 
differences or trends for AT and SBC children with respect to primary problem domain. Also, when 
considered independently, analyses of teacher and peer report did not reveal differences in primary 
problem domain based on group (AT or SBC). Analysis of children's self-reported level of adjustment, 
however, revealed a significant interaction between type of intervention (AT or SBC) and type of 
primary and secondary problem areas. Specifically, children who had Internalizing Problems as their 
primary and secondary problem areas reported similar gains in functioning, whether enrolled in AT or 
SBC. However, students evidencing primary and secondary symptoms of externalizing disorders made 
significantly more gains in self-reported adjustment when enrolled in SBC than when offered AT. 
Additionally, students with primary and secondary problem areas representing characteristics of both 
internalizing and externalizing difficulties made greater gains in AT than SBC services. 

Finally, we examined gender and age differences in our data. No evidence for Sex x Group interactions 
emerged. However, our data revealed a significant Age x Group interaction which suggests that children 
10 years and under, but not adolescents, enrolled in AT report greater improvement in functioning than 
children in SBC. 
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Implications 

These data represent a small part of the big picture of the School-Based Counseling Project. We've 
focused exclusively on our psychopathology outcome measures. Some trends suggest improvement, 
while other findings highlight the lack of differences. These data were consistent with naturalistic 
assessments of children's mental health services documenting little change in targeted behaviors. One 
year of service provision might not be enough, given the severity of these children's problems. Perhaps 
findings after two years of intervention will provide a more positive picture. 

In addition to psychopathology indicators, we will examine school-wide indicators of adjustment (e.g., 
grades, disciplinary measures, absences, restrictive educational placements) which may be linked in 
important way to the availability of an on-site mental health counselor. We will also scrutinize the 
activities of our school-based counselors. For example, some clinicians favor a more cognitive 
behavioral style, others favor a more play-therapy focus to treatment; some more actively consult with 
school personnel, where others work more exclusively with the children. 
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Introduction/Purpose 

The research demonstration project. Outcomes of Parent- and Provider-Designed Systems of Care 
(CMHS 5HD5SM50736), is discussed in detail in Armstrong, Evans, Tannen, and Scudder (1995) and 
in Evans, Armstrong, Thompson, and Lee (1994). Two rural counties in upstate New York have been 
provided equal funding to compare the processes and outcomes of systems enhancements based on very 
different principles. In Essex County, the naturally occurring experiment of Families First provides a 
demonstration of the principle of parent involvement taken to its logical conclusion, being an 
organization designed to build a partnership between families and professionals and to develop services 
which respond to families stated needs. Herkimer County, on the other hand, has developed a 
provider-designed, school-based, family-therapy enhancement to their mental health services for 
children. 

Families First under the guidance of Naomi Tannen, its director, has been providing a broad range of 
services to the community for over two years. Services include phone support, resource library, respite, 
financial and transportation assistance, trainings and workshops, and social events. Families First 
employs two half-time Intensive Case Managers (ICM) who also work as family advocates, and a 
home-care provider. In addition, many parents are employed in a wide range of roles. Families First is 
constantly working at both the case level and the county administrative level to convince other agencies 
to involve families in decision-making, with varying levels of success. The research discussed in this 
presentation provides detailed examples of this work. 
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Method/Procedures 



Three methodological approaches are being used to assess the processes and outcomes in the two 
counties: a treatment outcome study, a network analysis study and a combination of qualitative 
methodologies. The qualitative research presented here was conducted in Essex County by the first 
author, and involved comparing two service participants' experiences in order to explore Families First's 
progress in family involvement and interagency collaboration. These histories provide examples of the 
practical ways in which agencies choose or are required to work with each other, often requiring 
face-to-face discussions and documentation in which opinions are openly expressed. By looking at 
interagency collaboration, we can assess the position of agencies vis-a-vis Families First as this new 
agency seeks to insert itself into the traditional system of agencies and as it seeks to change the attitudes 
and behaviors of those entrenched agencies. 



Severe school refusal was chosen for this study from the various problems of the more than 30 children 
and families helped by Families First's ICM program, because, almost uniquely, clarity of assessment of 
successful treatment is possible; the youth returns to daily school attendance. The two families were 
selected because both the students, Christie and Darlene (pseudonyms), and their family situations had 
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Strong similarities, because they were introduced to Families First at approximately the same time, and 
because their experiences of the system of care were very different. 

In-person, in-depth interviews were conducted with family members, with the Intensive Case Manager, 
and with many other service providers and administrators, as well as through phone conversations. 
Meetings at Families First and conversations with staff members kept the researcher abreast of 
developments. Where Darlene was concerned, interagency meetings and a court hearing were observed. 
Documentation was gathered from many sources. 

Two Cases Of Severe School Refusal 

Christie's and Darlene's circumstances had much in common superficially. Christie was 12 and Darlene 
14. Both lived in small family units in rural towns with large extended families close by. They were 
academically successful when attending school. Both not only refused verbally to go to school but 
fought violently not to be taken inside, and were out of school almost every day of fall 1993. Both 
schools were obliged to file a PINS petition or be able to show reason for not doing so. Both families 
were reported for child abuse by neighbors who saw that the girls were not going to school and that 
furious arguments were going on. Both mothers blamed themselves in part for the defiant behavior. 
Darlene's family seemed to have more social and cultural capital (Bourdieu, 1986) than did Christie's. 
Darlene's mother had been a foster mother for seven years. Her older sister had been a model student. 
Though never having lived with them, Darlene's father maintained a positive supportive relationship 
with Darlene and her mother. A retired corrections officer and successful business owner, his two sons 
were high-school graduates. Realizing the school avoidance problem was too much for her, Darlene's 
mother drew on her connections in the town and the county for help, to get Darlene to school, and to get 
a psychiatric evaluation and appropriate treatment. 

Christie's family, on the other hand, was under great stress: relatives' illnesses, their recent moving, and 
her father's unemployment and protracted workman's compensation case. They had no parent/child roles 
and no family routine. Christie's parents had become as physically withdrawn as Christie and rarely left 
the house. Neither parent had much schooling. There was little contact with the community outside the 
extended family. 

Parent Involvement and Interagency Collaboration Efforts 

The school referred Christie and her family to the mental health clinic, whose staff introduced them to 
Families First. Families First and the school psychologist convinced the principal and school board of a 
psychiatric problem, providing time for evaluation by a psychiatrist and for modeling of normal 
parent/child roles and routines by Families First's home-care provider. When Child Protective Services 
were called in, they agreed an evaluation was needed and agreed to split the costs with the school and 
Families First. The psychiatrist found by the school psychologist was respectful of and encouraging to 
Christie's family; her practical recommendations laid the groundwork for a treatment plan designed 
collectively by family members and agencies. Positively-reinforced incremental steps were designed and 
carefully attended by the ICM and the school psychologist to reintroduce Christie to school, where 
changes were made to provide welcome and support. Once Christie was established in a school routine. 
Families First shifted focus to helping her parents become independent of the family. Christie's mother 
now works as a parent advocate at Families First. Christie went to school all through fall, 1994 and 
winter, 1995. 



Darlene's school had less patience for her resistance to school. Threats of removal from home had 
succeeded with other students, so they filed a PINS (Person in Need of Supervision) petition in October 
1993 which led to a court-ordered 30-day residential evaluation and the shifting of Darlene's custody 
from her mother to Social Services. Darlene's mother's prior requests for help from Social Services, with 
whom she had worked as a foster parent, fell on deaf ears. The evaluation blamed the parents and 
recommended long-term residential treatment. Families First became involved while the family 
appealed, working with Darlene's family and the school to implement a desensitization treatment plan, 
made in collaboration with a child psychologist, an expert on school refusal, hired by Families First. 
Having made curriculum changes, the school had little tolerance for less than total attendance. Darlene's 
problems took a toll on Families First: much extra time was needed to talk and meet with lawyers, and to 
help with violent outbursts at home, as well as the expected difficulties of implementing the treatment 
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plan. 

Attendance having been up and down, the court decided that Darlene should live with her father while 
awaiting residential placement. Darlene's responsible behavior over the summer impressed Families 
First but not Social Services, who placed her in a facility an hour from home. In September 1 994, 
Darlene revealed she had been sexually assaulted at her home school. For Families First and their 
consulting psychologist this news explained her school avoidance. Her father increased his efforts with 
the courts to get her home. But Darlene then became pregnant and this was seen by Social Services as 
another indication of incompetent parenting. Despite the family's and Families First's efforts to negotiate 
her return home. Social Services moved Darlene in January to a facility for pregnant girls further from 
home where education was not a priority, despite its being the motivating factor in the initial placement. 
Finally, Darlene's father's home was evaluated and in March the court gave custody to her father. 

Return to Top 



Conclusions 

The striking difference between the following descriptions illustrates the gap between alternative 
solutions to the problem of school phobia. The Social Services lawyer characterized Darlene's family as 
intergenerationally dysfunctional : 

Her familial history suggests that she will simply be another teenage mother without a high school 
education, with parents who are dysfunctional, who have raised a dysfunctional child, and now 
want to help raise a dysfunctional grandchild. 



This view of Darlene's family as essentially a bad influence is completely the opposite of the view of the 
consulting psychologist: 

Darlene has been alienated from her family by the abuse of a stranger and her fear of revealing this 
secret. It seems to me only a further exacerbation of this wound to keep Darlene, at this point, 
from her family&shyp; a family who wants to nurture her, by whom she wants to be nurtured, and 
who, until she was so cruelly and secretly misused, was able to provide her totally adequate care. 

It is inconceivable to me that she could be transferred to an unfamiliar environment at even greater 
distance from her home when her parents are so willing to provide for her and her unborn child. 



Definition of the problem and the range of possible actions are tightly shaped for service providers by 
their agency's mandate and history; some agencies welcome family involvement in planning solutions 
while others do not. Through dealing with Christie's and especially Darlene's problems. Families First 
has learned much about how to build successful collaborations so that residential placement is agreed to 
be the last resort. 

Some, though not all, of the variables which contributed to the extension of the trials and tribulations of 
Darlene and her family are related to agency collaboration efforts: 

1. Agency knowledge of new services can be important: perhaps ICM may have been too new a 
concept for Social Services and the judge, and perhaps now Families First and the ICM program 
will be given a second chance to show what can be accomplished at home. Social Services appears 
to remain skeptical and further interviews will provide valuable insight into their perspectives. 

2. The school's experience will also be telling. School perspectives on school refusal can shape 
crucial moments, such as non-attendance being considered truancy or an emotional problem. 

3. Lawyers' disregarding the importance of interagency collaboration may be detrimental to 
collaborative efforts when the courts are involved, increasing conflict among involved agencies. 

4. Shifts in the personnel can alter the atmosphere: in this case, change of lawyers may have made 
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things more difficult, change of judges may have facilitated Darlene's going home. 
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Introduction 

This study sought to answer several pertinent questions about the Lincoln Public School (LPS) 
Behavioral Skills Program (BSP). First, the study looked at whether students referred to 
BSP exhibited a level of behavioral and emotional problems and experienced levels of psychosocial 
stressors which was higher than the general population of students with Behavioral Disorders (BD). 
Second, this study sought to document the outcome of treatment by measurement of change in academic 
achievement, behavioral assessments, and performance of behavioral goals. Third, this analysis 
attempted to determine if students in this program had relatively more success in certain types of 
behavioral goals compared to other types of goals. 
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Method 

Participants 

Thirty students identified as having a BD, in grades Kindergarten through 9, enrolled at BSP during the 
1993/94 academic year were included as participants in the study. Males made up over 97% of the 
participant group. Students of European descent made up 

68% of the participant population, 24% were of African American decent, and 8% were of Native 
American heritage. The participants had been unsuccessful in less restrictive treatment alternatives, and 
74% had a history of psychiatric hospitalization. 

Procedure 

The Child Behavior Checklist-Teacher Report Form (CBCL-TRF; Achenbach & Edelbrock, 1991) was 
completed by home school teachers and BSP teachers at the beginning of the academic year 
or shortly after admission to determine initial levels of problem behaviors. Total problem scores, as well 
as externalized and internalized behavior scores, were compared to CBCL-TRF ratings collected by 
Conoley &. Peterson (1989) on a state-wide sample of children verified as BD served in regular schools 
in Nebraska. 

School records, court records and other official documentation were collected to determine the number 
and type of psychosocial stressors students had experienced. This information was compared to research 
findings documenting the number of stresses associated with severe maladjustment and with similar data 
collected from other samples of students with BD in Nebraska and in other published research. 

Academic achievement was assessed using a pre-post comparison of performance on the Kaufman Test 
of Education Achievement-Brief Form (KTEA; Kaufman & Kaufinan, 1985) in reading, spelling, 
mathematics, and composite scores. The test was administered at the beginning of the academic year or 
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shortly after admission and again at the end of the academic year. 

Behavioral change was assessed by making a pre-post comparison of CBCL-TRP's completed at the 
beginning of the school year or shortly after admission and the results of a second set of ratings taken at 
the end of the academic year by both home school and BSP teachers. Behavioral change was also 
assessed by tracking the change of three behavioral goals from each participant's Individual Education 
Plan (lEP). These goals were continually tracked during each class period at BSP. A zero tolerance level 
was maintained. If the child, at anytime in the class period, did not achieve the goal, it was marked as 
not successful. Daily averages on these goals were converted to 8 week averages, calculated four times 
over the course of the school year. 

The success students exhibited on the various types of goals was analyzed to determine if there was any 
differential progress. lEP goals were coded based on a manual of lEP goals used by the Lincoln Public 
School District. The goal categories included classroom skills, mainstreaming and adjustment, self 
esteem and self concept, coping skills, peer and adult interactions, and personal responsibility. 

Treatment 

The Behavioral Skills Program (BSP) is a collaborative program conducted by Lincoln Public Schools 
and Lincoln/Lancaster Child Guidance Center in operation since 1986. The program provides 
therapeutic services in home schools and intensive therapy and educational services in a self-contained 
site that students are bused to daily. Generally, students attend an hour or two a day at the home school 
and add time as appropriate for the student. The program is designed to focus on academic and 
behavioral skills development and the treatment of critical mental health issues. 
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Results 



The results of the initial CBCL-TRF and the level of psychosocial stressors BSP students experience 
indicated that BSP students were significantly more disturbed than the general population of students 
with BD. When both internalized and externalized ratings were included, 69% of BSP students had 
behavior problem levels in the clinical range. The CBCL-TRF scores provided by home school and BSP 
teachers were significantly correlated (.55 -.51). 



When compared to scores of a state wide sample of students verified as BD, it was found that BSP 
students problem behavior ratings were significantly higher. BSP students received a mean total problem 
score of 68. The mean rating of externalized behaviors for BSP students was 69 and the state wide 
sample was 60. Internalized behavior scores for BSP students were 64 compared to 50 for the state wide 
sample (Conoley & Peterson, 1989). 

Students served by BSP experienced high levels of psychosocial stressors. Previous research indicates 
that the presence of 3 psychosocial stress factors put children at risk of severe social maladjustment 
(Siefer, Sameroff, Baldwin & Baldwin, 1 992). The data collected indicated 

that BSP students experienced a mean of 10.6 psychosocial stress factors. Secondary students (grades 
6-9) had a mean of 12 stress factors and elementary students (grades K- 5) had a mean of 9 stress factors. 



This study addressed the question whether students at BSP demonstrated significant positive change in 
academic achievement. Overall, students made significant improvement in all three achievement 
subtests of reading, spelling, and mathematics, as well as in composite scores. Not only do these results 
indicate students were maintaining normally expected achievement progress, they were making marked 
increases in academic achievement. In addition, total risk positively correlated (r = .45) with academic 
achievement test improvement. In other words students entering with high degrees of psychosocial 
stressors showed the most improvement in academic achievement scores. 

The CBCL-TRF was used to detect significant change in teacher rating of the behavior problem level of 
students over the course of the year. The BSP teachers indicated that, overall, students showed 
improvement in total problem scores and internalized scores, and no improvement in externalized 
behaviors. Elementary student ratings showed more improvement than secondary students. 
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Specifically, elementary students' total problem scores decreased by 4 points and externalized behavior 
scores decreased by 3 points. Internalized problem behaviors showed a significant drop of 5 points. 
Home school teacher ratings of both elementary and secondary students showed a .25 increase in total 
problem scores, a 3 point decrease in internalized behavior, and a 2.3 increase in externalized problem 
scores. Overall, no significant deterioration in behavior scores was evident. 

Assessment of behavioral change was also documented by tracking lEP goals. There were 
18 goal comparisons possible, and of these 13 comparisons showed improvement and 5 showed 
significant improvement. Five comparisons showed no improvement or a slight decrease. However, no 
comparison of lEP goals showed a significant decrease. 

Documentation of progress on lEP goals indicated that students made immediate improvement in 
behavior. Behavior changes plateaued over time, but the plateaus were much better than the baseline 
levels of performance. Overall, BSP students achieved lEP goals 80% of the time in the program. This 
compares favorably with students with BD studied by Conoley & Peterson (1989) which found that 
students with less severe disturbances exhibited on-task behavior 78% of the time. 

A final question was whether BSP students made differential progress on various types of behavior 
goals. The results showed that BSP students were relatively more successful on classroom skills goals 
and self esteem goals over other types of goals. 
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Discussion 



It is evident that students referred to BSP exhibit a significantly more severe level of problem behaviors 
and individual and family psychosocial stressors than the general population with BD. Students of BSP 
experienced high levels of psychosocial stressors that were associated with significant maladjustment. 
These results provide a quantified measure that BSP students exhibit a level of behavioral and emotional 
dysfunction to warrant the level of intervention provided by the program. 



It is of particular interest that students at BSP did exhibit significant improvement in academic 
achievement. This is a remarkable finding in the context of previous research indicating that, in general, 
day treatment programs do not do a good job in supporting such gains. This result may confirm the view 
that children with BD respond best to treatment environments that provide opportunity for academic 
success, that reward prosocial behavior, that provide positive relationships with adults who model and 
train prosocial behavior and values, and that focus on student strengths and adjustment to the educational 
environment (Goldstein, Harootunian & Conoley, 1994; Knitzer, Steinberg, & Fleisch, 1990). 

Performance on lEP behavior goals proved to be a more responsive indicator of behavior improvement 
than the CBCL-TRF. As a result, changes of teacher ratings of problem behaviors were small or not 
significant in most cases. It should be noted that the comparison of problem behavior ratings did not 
indicate any significant deterioration. Given the severity of problems and psychosocial stressors the 
participant population experience, the lack of deterioration of problem behavior ratings may be seen as 
an indicator of positive progress. 



BSP is an environment which focuses on development of trusting adult/student relationships, that 
maintains safety, that can provide extensive amounts of time to resolve behavior problems, and where 
there is a persistent pursuit to find a way to teach each child. The improvement in lEP goals in the 
category of classroom skills and self esteem suggest that BSP is an environment that can manage 
aggressive behavior and at the same time give these children an opportunity to succeed and learn to self 
manage their behavior. Elementary students made the largest gains in both academic and behavioral 
measures, indicating early intervention is an important treatment consideration. 

The implications for future study are that results and continued data collection can be used to determine 
what program components are most effective and to assure that the program continues to maintain its 
success in developing academic and behavioral skills for children with severe levels of BD. 
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Introduction 

Research indicates that children from families with substance abuse issues suffer increased rates of 
social and emotional problems. Sometimes, though not always, these problems emerge in the form of 
behavioral acting out in school. School persormel are often unaware of the child's family circumstances 
and the child is labeled as "the problem." Even in those cases where familial substance abuse is known 
or suspected, resources to adequately address the child's needs are lacking. As such, children fi'om 
families with substance abuse issues may advance through school acquiring increasingly poor academic 
and behavior records without needed intervention. 

Some children from families with substance abuse issues, particularly girls, internalize their fears, 
fhistration, loneliness, and anger, adopting a more isolated, withdrawn presentation. These children are 
rarely noticed by school persormel at all and their psychosocial problems are likely to be entirely 
unaddressed. 

Substance abuse is a major contemporary social problem, particularly in America's irmer cities, where 
treatment resources are few. Consequently, children living in low-income, irmer-city communities are at 
particular risk for the psychosocial problems attendant upon familial substance abuse. There is an urgent 
need for services for these children; public schools provide an opportune location for such intervention. 

This paper describes the development and testing of a structured school-based psychoeducational group 
intervention designed to address common psychosocial difficulties presented in school settings by 
children who experience familial substance abuse. This two-year project was funded by a grant from the 
New York Commimity Trust to the first author. Data presented are based on first year findings. 
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Method/Procedures 

Sites/Subjects 

Three irmer-city Philadelphia elementary schools, selected for their locations in communities known for 
high rates of drug trafficking and drug-related crime, participated in this project. During the project's 
first year, eight treatment groups were conducted in two of these schools, one located in 
a predominately African-American low income community in West Philadelphia, and the other in a 
racially and ethnically diverse neighborhood of poor and working class families. 

In each participating school, project staff presented an informational session on substance abuse to all 
third through fifth grade classes. Parents were notified that substance abuse education would be taking 
place in their child's school including small group discussions; they were requested to contact the school 
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principal if they did not wish their child to participate. At the end of each classroom presentation, 
children who wished to discuss additional worries or concerns about substance abuse in their families or 
neighborhoods were invited to sign up to participate in a series of small groups to be held in the school. 
The names of children indicating a desire to participate were reviewed by school guidance personnel and 
those with classroom behavior problems and/or whose families were known or suspected of being 
substance-involved were targeted. One hundred eight subjects were selected in the two schools. 

The 50+ study subjects in each school were randomly assigned to two groups; half received the 
intervention in the Fall semester and the other half in the Spring, thereby forming a naturally occurring 
control group for subjects participating in the Fall groups. 

Intervention/Program 

The intervention was designed to follow the format of a widely used, highly structured curriculum for 
groups of children of parents with alcoholism entitled, "Children Are People, Too." This curriculum uses 
an 8 to 10 session treatment model, each session following a similar format with repeated opening and 
closing exercises. 

A specific issue relevant to the psychosocial concerns of children who have family issues of alcoholism 
is addressed in each session. Topics included: identifying and appropriately managing feelings about 
parental addiction, personal safety issues, imderstanding substance abuse, how to interact with peers, and 
others. 

Because the "Children Are People, Too" curriculum was designed to be used in conjunction with 
addictions treatment for a parent with alcoholism and presupposes the support and involvement of a 
second, parent who does not drink, many aspects had to be revised for a population where children often 
live with a single parent who is frequently unable to address his/her addiction. Another significant 
difference was that most children in our sample were concerned with the effects of crack-cocaine and 
other illicit drugs on family members, rather than alcohol dependence, although many times these 
co-occurred. Drug-related violence both within the family and in the neighborhood was also a primary 
concern of our subjects and is unaddressed in the "Children Are People, Too" curriculum. 

Finally, the "Children Are People, Too" curriculum relies heavily on the verbal ability of participants. 
Our subjects tended to have limited verbal skills and, thus, were highly dependent on other forms of 
self-expression such as drawing, singing, and story-telling. Also, because many of the children had 
learning disabilities, attention deficits, and problems with impulse control, we foimd it necessary to 
make significant adaptations to the curriculum to accommodate the special needs of these children. 

Groups were held once a week during school hours for 1 0 consecutive weeks during the fall and spring 
semesters. Each group session lasted one hour and followed a highly structured format. Pre- and 
post-testing were done at the first and last group sessions. 

Measurement/Instruments 

All subjects were tested at three points in time: in early Fall before any groups were held, in late Fall 
after the first round of groups, and in late Spring after the second roimd of groups took place. Data were 
gathered on both experimental (Fall group participants) and control subjects (Spring participants) in this 
way. 

Three standardized self-report instruments were used. Each instrument was selected to reflect an issue 
found in previous research or in clinical observation to be of particular concern to children from families 
with substance abuse issues. Instruments used included: the Nowicki-Strickland Locus of Control 
Questionnaire (Nowicki & Strickland, 1973); the Children's Loneliness Questionnaire (CLQ; Asher & 
''feeler, 1985); the Self-Perception Profile for Children (Harter, 1985). Data on subjects' classroom 
performance and behavior were collected from teachers using the Teacher's Report Form (TRF) of the 
Child Behavior Checklist (Achenbach & Edelbrock, 1986). Group leaders also rated participants with 
regard to their perceived levels of comprehension of the key concepts contained in the curriculum. 
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Results and Implications 

Findings from the first year of this project, though limited because of the small sample size (48 
experimental subjects and 48 controls completed treatment and data collection) suggest that a structured 
psychoeducational group intervention of relatively brief duration aimed at children from families with 
substance abuse issues can help to decrease inappropriate classroom behavior and other social 
difficulties. Teachers' reports on the TRF indicated significant changes in several areas of classroom 
behavior, including peer relations, interpersonal aggression, and disruptive conduct. 

Group leaders' ratings of participants' comprehension of group concepts illustrate the importance of 
adapting a standard curriculum to the needs of the population treated. Slightly over half of the children 
(N= 56) who participated in the intervention during the first year understood half or fewer of the 
concepts presented. Just forty-five percent (N= 47) understood most or all concepts presented. 

We hypothesized that a child's level of comprehension of concepts would affect the 

degree of change that took place on the independent variables measured including self-concept, 

loneliness, locus of control, and classroom behavior. However, analysis of variance found no significant 

association between level of comprehension and the degree of change reported on these dependent 

variables. 

Finally, findings from the self-report instruments were mixed. The Children's Loneliness Questionnaire 
found higher than average levels of loneliness among the subjects in this study&shyp;a finding 
consistent with the clinical literature on the effects of parental substance use on children. On the other 
hand, locus of control scores for study participants were close to published norms for same-age children, 
an unexpected finding given the literature which suggests that children from substance-involved homes 
are more likely to demonstrate externalized locus of control. Similarly, mean scores on the subscales of 
the Self-Perception Profile for Children were very close to normative means published by the 
instrument's author (Harter, 1985). 

T-tests were carried out on each of the dependent variables to determine if there were significant changes 
on any of the dimensions assessed. Only the Children's Loneliness Questionnaire showed significant 
pre- and post-test differences, with significantly lower scores post- treatment for the treated group 

(N = 43; T = 2.3421; p = .02). This finding suggests that children who participated in the intervention 
achieved one of the project goals: to decrease social isolation and connect with other children with 
similar experiences with familial substance abuse. 

Additional data are currently being collected during the second project year which will allow for more 
data for analysis and greater confidence in study findings. 
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Introduction 

Many children with emotional/behavioral disturbances are adjudicated dependent, remain in the foster 
care system for years, experience frequent transfers from one placement to another, and make little or no 
progress toward reunification or adoption, or toward improved emotional/behavioral adjustment. The 
Fostering Individualized Assistance Program (FIAP) was developed to provide individualized 
wraparound supports and services to foster children with emotional/behavioral disturbances and to their 
families (i.e., foster, biological, and/or adoptive). The outcomes hoped for were a decrease in the rate of 
placement changes, improvement in emotional/behavioral adjustment, and an increase in the likelihood 
of family reunification or other permanency arrangements. 

The FIAP intervention was evaluated in a controlled, random-assignment study, which compared a 
sample of at-risk children receiving the individualized, wraparound process (FIAP group, n = 54) with a 
comparable sample of children experiencing practices that are standard in the foster care system (SP 
group, n = 78). The initial demographic variables, scores on certain normed instruments, and history of 
family and child risk factors, are presented in this article to evaluate equivalence of the two randomly 
established groups (FIAP and SP) to each other, and to the populations of children with 
emotional/behavioral disturbances found in other major studies. 
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Methods 

Subjects and Settings 

The target population was children in foster homes or group shelter care at the start of the study who 
had, or were at risk of having, emotional/behavioral disturbances (EBD). Foster care caseworkers 
completed screening checklists of risk factors identified in previous research (Boyd, Struchen, & 
Panacek-Howell, 1989) for children in their caseloads between the ages of 7-15, without mental 
retardation. The "at risk" pool was identified as those children having at least three major risk factors, 
and 132 children were randomly assigned from this at risk pool to either the FIAP or the SP group. 
Return to Top 



Data Sources and Analyses 

All children and their caregivers in both groups were interviewed in their homes every six months by 
trained interviewers blind to the group assignments. Both normed instruments such as the Child 
Behavior Checklist (Achenbach & Edelbrock, 1983) and structured interviews were used. Additional 
data was obtained from computerized records of foster care placement and delinquency histories, case 
records, and school records. Differences between groups were tested by Chi-square analysis, and one- or 
two-way ANOVAs, as appropriate to the data. 
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Results 



Severity and Equivalence of FIAP and SP Groups 

There were no significant differences between FIAP and SP groups in the proportion of males vs. 
females, Whites vs. non- Whites, or in the number of children in each of three age groups rSee Table U . 
Also, there were no significant differences in the average age of children in FIAP vs. SP groups (1 1.8 vs. 
11.6 yrs.), time in foster care prior to entry into the study (31.6 vs. 30.7 months), or number of 
placement changes in foster care, per child per year, prior to the study (3.7 vs. 4.2). However, youth ages 
13-15 had significantly higher residential placement change rates (6.02 and 6.50) than the other two age 
groupings (f = 18.8, df = 2,126, p = .000). 

All scores on both the CBCL and YSR (Youth Self Report) were reported as T scores, with a mean of 50 
and standard deviation of 10, in order to combine results from both genders and all age groups. The 
FIAP and SP groups were not significantly different at entry into the study in total behavior problem T 
scores on either CBCL or YSR. This similarity between FIAP and SP groups also held for the 
externalizing syndromes on both instruments. The only significant difference between groups was for 
the internalizing syndrome on the CBCL, for which the SP group had lower T scores on the internalizing 
syndrome than the FIAP group (SP = 60.0, FIAP = 64.5; p = .02). 

Comparison with Other Populations of Youth with EBD 

A comparison of the FIAP study sample to the samples in each of two important national studies was 
made. The first study used was the 812 children in the National Adolescent and Child Treatment Study 
(NACTS; Silver, Duchnowski, Kutash, Friedman, Eisen, Prange, Brandenburg, & Greenbaum, 1992). 
Youth in the NACTS sample had all been identified by the school systems as severely emotionally 
disturbed (SED), and were receiving special educational services for SED, or were receiving publicly 
funded residential mental health services. 



A second comparison was made with a sub-sample of 87 youth from the Alternatives to Residential 
Treatment Study (ARTS; Duchnowski, Johnson, Hall, Kutash, & Friedman, 1993). Like NACTS, the 
youth in ARTS were all enrolled in programs considered to be "deep end" placements (primarily 
residential programs), and were all identified as having emotional disabilities. The youth in the FIAP 
study, compared to either the NACTS or ARTS samples, were in less restrictive placement settings, with 
none currently in mental health treatment settings and only 57% receiving ED/BD school services at 
entry to this study. 

As shown in Table 2. there were some demographic differences between the FIAP study sample and the 
NACTS and ACTS samples. By comparison, the FIAP study sample was younger (mean = 11.7, range = 
7-15 yrs), compared to the sample in NACTS (mean = 13.9, range = 8-18 yrs) and ARTS (mean = 14.2, 
range = 7-18 yrs). Also, compared to either the NACTS or ARTS samples, the FIAP study sample was 
less dominated by males (FIAP = 60%, NACTS = 75%, and 64% for ARTS), and had a larger 
proportion of ethnic minorities (FIAP = 39% minorities, compared with just 29% for both ARTS and 
NACTS). 

The FIAP sample had a mean total problem T score on the YSR of 61.1, almost exactly the same as that 
reported for the ARTS sample (62.2). On the CBCL, the total problem score mean was 67.4 for the FIAP 
sample, compared to 71.5 for ARTS, a difference of less than 0.5 SD. The CBCL externalizing 
syndrome scores for the FIAP study sample (mean = 68.2) were slightly, but not significantly, lower 
than those of school-based (mean = 69.8) vs. residential (mean = 72.9) NACTS children. CBCL 
internalizing syndrome scores for the FIAP study (mean = 61.8) were lower than that reported for both 
school-based and residential children in NACTS (mean = 66.6, and mean = 71.6 respectively). 



Using the same total problem score cutoffs used by ARTS to classify their sample into probably 
"normal" (T < 60) or "deviant" (see Achenbach, 1991 for discussion of comparison populations), 
caregivers rated 81% of the FIAP sample's behavior as deviant (compared to 98% for ARTS), and 59% 
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of the youth rated their own behaviors as deviant (compared to 66% of the ARTS youth). 

Family History Risk Factors for FIAP Children. 

At entry to the FIAP study, the average time in foster care for FIAP study children exceeded the 1 8 
months legally mandated by the State of Florida by over a year (FIAP = 31.6 months, SP = 30.7 
months), ranging from 3 months to 7.5 years. While in foster care, the average child experienced 4 
residential placement changes per year (excluding brief runaway episodes; FIAP = 3.7, SP = 4.2). There 
was a significant difference between the two study counties, (5.2 vs. 3.3 changes/year) with the higher 
rate occurring in the more urban county (f = 7.7, df = 1,130, p < .01), but there were no significant 
differences between the FIAP and SP groups. 

More of the FIAP study sample had experienced out-of-home placements specifically due to abuse 
and/or neglect reports in the period prior to the study, 72% of a subsample of 87 from one urban county, 
compared to 56% of the ARTS sample. Over 50% of that sub sample (n = 63) 

had records of 4 or more abuse investigations each (range = 2-14). Child protection investigations found 
that for only 17% of the youth were no specific indications confirming the abuse or neglect&shyp;no 
investigations were closed as having no indications. 

Many of the FIAP study children reported family and environmental factors that could suggest higher 
risk for emotional/ behavioral disturbances (See Table 3) . On the initial interview, 43% of the children 
reported one or more members of their family had serious substance abuse prob-lems, 54% reported a 
family member had been incarcerated, 21% reported a family member had a history of suicide attempts 
or psychiatric hospitalization, and 44% reported a parent with serious emotional disturbance (no 
differences between FIAP and SP groups were statistically significant across those variables; p > .05). 
Return to fop 



Conclusions and Interpretation 

It is apparent from the data presented and from other indicators such as high levels of involvement with 
the juvenile justice system (Lee, Clark, & Boyd, 1993), and use of psychotropic medications, that the 
children in the FIAP study showed high levels of EBD, close to that of other identified EBD populations 
(NACTS and ARTS). Although there was one significant difference on sub-scale scores of the CBCL 
and YSR between FIAP and SP groups, the preponderance of evidence on adjustment scores and other 
indicators shows close equivalence of the FIAP and SP groups to each other. Thus, it seems reasonable 
to conclude that the randomization to control and experimental groups was successful, as was the 
selection process for identifying a population of EBD children from a general foster care population. 
Return to Top 
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Tablet 

Group Demograph ics at Start of Study 



HAP £P 

«=K «= 78 

Ethxodty 



White 


31 


57% 


50 


64% 


Nbn-vAiite ^ 


23 


43% 


2S 


36% 


Gender 


Ma]e 


32 


60% 


48 


61% 


Feztole 


22 


41% 


30 


39% 


Age gjDuis 


7-Sy&ms 


16 


30% 


27 


35% 


10-12 years 


16 


30% 


23 


30% 


13-15 years 


22 


41% 


28 


36% 
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Table 2 

CBCL andY9=i Scores or Clinical Ranges 
Compared to Other EBD Populations. 
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ARTS 


NACTS 


T scores: 






School 


Residential 
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6S2 
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725 


Internalizing 


613 
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666 


71j6 


Tool RoUjemscooie 


67.4 


715 




— 


YSR 


Tool RoUjemscooie 


61.1 


622 
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%r1irriral 


81^ 


9^ 
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5S^ 


66^ 
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Tables 

Fam ily Risk Factors and Child History tor FI A P and SP Groups 




HAP 




SP 


Family xisk iacton 


» = 54 




« = 78 


Incaice latiD n 


27 5Q% 


44 


56% 


Histoiy of emotiDiialdistiiibsjice 


24 44% 


34 


44% 


Substance abuse 


22 41% 


35 


45% 


Psych.aiiic hospV^ridal 
Quid history factoos 


14 21% 


14 


21% 


Time out of home 


31j 6 mos 


307 mos 


Placement changes/yeai 


37 




42 


Uiban couTLty 


5.1 




53 


Ruxal county 


28 




3j6 


Abuse /neglect mvest^ations 
(iu:S7,u&amonIy) 


72% 




73% 


Mean in vest^ations/ chili 


45 




39 


^ciale duration 


49% 




62% 
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Overview 

Over the past several decades, foster care has been expected to be a major solution to the plight of 
children who were neglected, abused, or abandoned by their families of origin. However, despite many 
attempts toward meeting the mandates (e.g., permanency, preventing out-of-home placements) of the 
Adoption Assistance and Child Welfare Act of 1980 (P.L. 96-272), it has been widely asserted that the 
foster care system is a part of the problem besetting these children (e.g., Bryant, 1993). 

Despite evidence from providers and administrators of foster care services that over ninety percent of all 
children in out-of-home care ultimately return home ("Keeping families together," 1993), models of 
family-focused-permanency plaiming that include family preservation, reunification, and a 
family-focused therapy orientation have not been consistently incorporated into services for children 
residing in out-of-home placements. When family-focused programs of family preservation have been 
tried, improved child placement rates have been obtained. In a review of samples comprised primarily of 
child welfare cases, placement outcomes were improved with programs that emphasized therapeutic 
interventions with the family versus program models that emphasized community, and concrete services 
or crisis intervention (Nelson, Landsman, & Deutelbaum, 1990). 

There has also been a call for the children's service system to adopt alternative values and therapeutic 
interventions with children and families in order to provide for the protection, nurturance, and 
development of children and to ensure the integrity of their families (e.g., Boyd, 1992). The Fostering 
Individualized Assistance Program (FIAP) study was designed to implement, and to investigate the 
effects of, an individualized, case-managed, family-focused collaborative intervention with children and 
their families&shyp;natural, relative, foster, or adoptive&shyp;in the child welfare system (Clark & 
Boyd, 1990, 1992). This summary provides preliminary outcome data for a controlled experiment, with 
random assignment, with a group of children in foster care who received FIAP services and a 
comparable group who were supported by practices standard to the foster care system. More specifically, 
findings from analyses of children's wave 1 (time of entry into the study) through wave 7 (3 years after 
entry) Youth Self-Report (YSR; Achenbach, 1991) data using growth curve analysis (Bryk & 
Raudenbush, 1992) are reported and the implications of these findings are discussed. 

Return to Top 



Method 

Subjects 

Children in the state foster care system were eligible for inclusion in this study if they were (a) in 
temporary custody of the state, due to having been abused or neglected; (b) 7 years to 15 years old; (c) 
living in a regular foster home or in an emergency foster shelter facility; and (d) having behavioral and 
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emotional disturbances, or at risk of such, as defined by several behavioral screening indicators (e.g., 
potential harm to self or others, use of drugs and/or alcohol, engagement in abnormal sexual behavior). 
Children from this pool were selected using a computer-generated random-number system and assigned 
to the FIAP enhanced services group or the standard practice (SP) control group. To compensate for a 
predicted higher attrition rate within the SP group (e.g., due to extended runaways or voluntary 
dropouts), approximately 50% more subjects were randomly assigned to the SP group than to the FIAP 
group. A total of 132 foster children participated, 54 assigned to FIAP and 78 assigned to SP. 

Intervention Conditions 

Standard Practice Group (control group). The standard practice control group received the prevailing 
care, support, and services that the state system provides to children who have been adjudicated as 
dependent and placed in the foster care system. 

FIAP Group (experimental group). The FIAP group received services aimed at improving 
behavioral/emotional adjustment and placement stability, in addition to services received by the standard 
practice group. These additional services consisted of four major components: strength-based 
assessment, life-domain planning, clinical case management, and follow-along supports and services 
(Clark, Prange, Lee, Boyd, McDonald, & Stewart, 1994; McDonald, Boyd, Clark, & Stewart, 1994) . 

Data Collection and Instrumentation 

Data on behavior problems were collected through interviews, conducted at six month intervals for three 
years (seven waves of data including the first data collection). The data analyzed for this summary were 
provided by the youths themselves who completed the Youth Self-Report (YSR; Achenbach, 1991). 
Specifically, we analyzed the YSR Total Problems Scale which 
was composed of eight problem subscales: 

(1) withdrawn, (2) somatic complaints, 

(3) anxious/depressed, (4) social problems, 

(5) thought problems, (6) attention problems, 

(7) delinquent behavior, and (8) aggressive behavior. 

Analysis 

The specific analyses performed consisted of a sequential progression of analyses using two through 
seven waves of YSR data with treatment condition, age, and gender as second- stage or between-subject 
factors. The first analysis utilized two waves of data, comparing subjects change between waves 1 and 2. 
For each subsequent analysis, one wave of data was added (e.g., the second analysis, utilizing three 
waves of data, compared subjects change for waves 1 through 3). This sequence was followed until the 
final analysis, which utilized all of the current data collected on subjects. Growth curve analysis was 
used for all analyses except for the analysis of change between waves 1 and 2 which was done using a 
repeated measures ANOVA. The necessary minimum of three waves of data precluded the use of growth 
curve analysis in this instance. 



Growth curve analysis, known as hierarchical linear modeling, random regression modeling, random 
effects modeling, "multi-level" modeling, and empirical Bayes modeling, was the primary statistical 
method chosen for analyzing data. The growth curve approach offers several major advantages 
compared to more traditional means for analyzing longitudinal data such as repeated measures ANOVA. 
Specifically, (a) growth curve analysis places explicit focus on how individuals change in addition to 
how groups change, (b) growth curve analysis is flexible with regard to the treatment of missing data; it 
is not necessary for subjects to have data across all waves of collection in order to be included in the 
analysis, and (c) the assumptions underlying the growth curve approach are more realistic than those 
underlying more traditional approaches (Bryk & Raudenbush, 1992; Gibbons et al., 1993; Hedecker, 
1993). 



Growth Curve Analysis. In the growth curve framework, change is represented in a two-stage model 
encompassing both within- and between-subject factors (Bryk & Raudenbush, 1992). In 
the first stage, each person's observed scores are modeled as a function of an individual growth function 
(i.e., within subjects) plus random error. When these growth functions, which may be linear or 
non-linear, are applied to each individual's observed scores, they produce trend patterns for each 
individual subject (each subject's personal change pattern). These individual trend patterns are outcomes 
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to be explained in the second stage of analysis. In the second stage, the parameter estimates (e.g., the 
slopes) describing the individual trend lines, are tested for variation as a function of group differences 
(i.e., between subject factors such as treatment condition, gender, and age^ 
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Results 

Results of the analyses showed that scores from both groups of subjects decreased significantly over 
time indicating improvement in emotional/behavioral adjustment (e.g., for the wave 1 through wave 7 
analysis, t = -6.41, p = .000). In addition, a significant treatment effect (F = 3.95, p = .049) in favor of 
FIAP was shown for the period between waves 1 (entry into study) and 2 (6 months later). This main 
effect of treatment, however, was not present in results of the analyses from later waves. Instead, results 
of all analyses from waves 1 through 3 up to waves 1 through 6 indicated that a marginally significant 
interaction for the effect of status by age by gender on individual change patterns was present. This 
interaction was found to be significant in the wave 1 through 7 growth curve analysis discussed in detail 
below. 

Wave 1 Through Wave 7 Growth Curve Analysis 

In the growth curve analysis for waves 1 through 7, we included an intercept parameter (i.e., where 
children start out) and a linear slope parameter (i.e., children's linear change over time) to model 
children's individual-level YSR change patterns. Additionally, it was made apparent through a series of 
preliminary tests and procedures, including the plotting of individual's YSR scores over time, that while 
many children exhibited linear patterns of change, many others exhibited curvilinear patterns of change. 
Therefore, a quadratic parameter was also used to model individual-level YSR change patterns. A 
sample of the plots used to determine the parameters employed in the level- 1 model is presented in 
Figure 1. Subject A in Figure 1 shows a linear change pattern while Subject B's change pattern is 
curvilinear, indicating the need for both linear and quadratic parameters. 

Results of the first stage of this analysis (i.e., specifying the model of individual change patterns) 
indicated that significant variation existed in individual change patterns. Results of the second stage of 
this analysis (i.e., examination of individual change patterns in relation to group-level variables) showed 
a significant interaction for the effect of status by age by gender on these individual change patterns (t 
=-2.24, p =.025). 



This interaction indicates that efficacy of the FIAP intervention depends on the subjects' age and gender. 
For older FIAP males and younger FIAP females, there was a significantly greater decline in YSR 
scores compared to their SP counterparts. 

Return to Top 



Conclusion 

Results of the analyses showed that the FIAP intervention accelerated positive change in 
emotional/behavioral adjustment for children between waves 1 and 2, and this greater rate of positive 
change endured for older FIAP males and younger FIAP females, relative to their SP counterparts. 

From a systems perspective, older adolescents were more likely to have children«feshyp;and while 
females are usually encouraged to remain in the foster care system for the benefit of their 
child«feshyp;their male counterparts may be more likely to be promoted to live independently. Because 
the FIAP intervention was specifically designed to provide support and encouragement for the transition 
from foster care assistance to independent living, (or to obtain stable placements within the foster care 
system), it makes some clinical sense that older FIAP males would show greater improvement compared 
to their standard practice counterparts who may not get this support. The significant impact for younger 
FIAP versus standard practice females appears to have no direct ties to the specific type of supports 
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given by the FIAP intervention. Possibly this reflects the over-representation of female family specialists 
who bonded more readily with same sex younger 
FIAP youth. 

These small effects regarding clinical adjustment outcomes may be understandable in light of the 
complexities involved in use of a non-intensive therapeutic intervention to improve clinical adjustment 
with extremely challenging children and their families (e.g., Bickman, 1993; Cross, Fallon, Gardner, 
Adnopoz, & Saxe, 1992). As wraparound systems of care strategies are refined and ameliorated, as 
family systems therapy is incorporated more adequately (Henggeler, Melton, Smith, & Schoenwald, 
1993), and as personnel selection, clinical training, and field supervision methods are strengthened, the 
fidelity of the individualized wraparound approach should be improved. 

In addition to these results, it should be noted that the impact of FIAP on placement- 

related outcomes has been substantial (Clark, Lee, & Prange, in press; Clark, Prange, Lee, Boyd, 

McDonald, & Stewart, 1994). Although tentative at this time, the interim outcome results suggest that: 

(1) FIAP children are significantly less likely to change placements than are those in the SP group; and 

(2) FIAP children are significantly more likely than SP children to be in permanency settings with their 
parents, relatives, adoptive parents, or living on their own. Data also suggest that FIAP children in 
permanency placements tend to show better emotional/behavioral adjustment than do SP children in 
permanency placements (Clark, Prange, Lee, Boyd, McDonald & Stewart, 1994). Examinations of 
community adjustment indicators for subsets of children who have any history of runaways or 
incarceration, suggest that the FIAP children are showing trends for fewer days in both of these areas 
(Clark, Lee, & Prange, in press; Clark, Prange, Lee, Boyd, McDonald, & Stewart, 1994). A continuing 
increase in incarceration days and other deep-end placements by the SP group may ultimately 
demonstrate the FIAP intervention to be cost effective. 

This study also demonstrates some advantages of the growth curve approach for analyzing longitudinal 
data. The explicit focus on individual, as well as group change patterns, provides a conceptual focus not 
offered by more traditional statistical methods. The information provided by this approach on 
inter-individual variation in personal change patterns and differences in 

the shape of these patterns (e.g., linear versus quadratic), suggests improved conceptualization. Also, a 
greater number of subjects included and more realistic assumptions in the HLM analyses resulted in 
increased statistical power relative to more traditional analyses. 
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Introduction 

The Fostering Individualized Assistance Program (FIAP) was designed to study the outcomes of 
provision of highly individualized, wraparound services (the independent variable) versus standard 
practice for two randomly assigned groups of children with severe emotional disturbance in out-of-home 
foster care (FIAP group, N = 54; Standard Practice group, N = 78). 

For each FIAP child, one of four Family Specialists (FSs) was asked to do whatever was necessary, 
usually through teams of adults influential in the lives of their clients, to: (1) identify the strengths and 
needs of each child and the potential permanent placement family; (2) devise and implement a service 
plan capitalizing on both strengths and needs of each child and family (across 12 life domains); (3) 
monitor and record progress in each client's notebook; (4) advocate and trouble-shoot, often in 
innovative, creative and non-standard ways; (5) adapt and modify the plan as dictated by the constantly 
changing child, family, worker and system circumstances; and 6) follow along and support the client 
after permanent placement. 

Preliminary analysis of the first 30 months of data suggested that the FIAP approach was moderately 
successful: 

• FIAP children were significantly less likely to change placement than those in the Standard 
Practice (SP) group; 

• FIAP children were more likely to show significant initial improvement in emotional/behavioral 
adjustment, a gain matched by SP children at the 12th month of intervention; 

• both groups show significant levels of improvement in emotional/behavioral adjustment over 
time; and 

• FIAP children were significantly more likely to be in permanency settings with their parents, 
relatives, adoptive parents, or living on their own. 

• Additionally, the data suggest that FIAP children in permanency placements tend to show better 
emotional/behavioral adjustment than do SP children in permanency placements. Finally, 
community adjustment indicators for subsets of children who have a history of runaways or 
incarceration show that FIAP children accrue fewer days in these areas (see Lee, et ah; Rose, 
Prange, Greenbaum & Clark, this volume). 



It has always been the intent of the authors to evaluate, not only the effectiveness of the overall FIAP 
approach, but also what aspects of its individ-ualized methodology (the independent variable) appear to 
be most and least successful in meeting the needs of children and their permanency families. As the 
clinical application of FIAP neared its ending date, efforts were made to identify the ten children with 
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whom FIAP had been most successful (Top Group) and the ten with whom FIAP had been least 
successful (Bottom Group). Success, of course, related to how well each child was doing in relationship 
to permanency, number of placements, length of stay, school progress, absence of illegal or otherwise 
handicapping behaviors, and other highly individualized, clinical variables. 

Relevant factors for these two subsets of the experimental group are being analyzed in an effort to 
discern and describe what worked, what did not work, and why. Preliminary findings are relevant to the 
study's research questions and to the role and nature of independent variables in the field of applied 
human service research. 
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Method 



Context 

FIAP was established as a collaborative effort between the state/district offices of mental health and 
child welfare and the University of South Florida. FIAP, however, was administratively separate from 
these offices and was permitted an adjunctive role, without being assigned control over either mental 
health or child welfare services for its clients. Thus, FIAP FSs provided a type of co-case management, 
with the regular foster care counselors already assigned to these children, who maintained primary legal 
and service relationships with the children and families. This proved to be reasonable workable and 
successful in some cases, and almost impossible in others. Family Specialists attempted to do necessary 
assessments, create the teams, devise, implement, monitor and track individualized service plans within 
the prevailing system. 

The real world result was the formation of somewhat loosely knit teams, per child, co-case managed by a 
FIAP FS and the assigned foster counselor. In many cases, the FS role became primary, in light of the 
overburdening active caseloads of each state foster counselor (e.g., 40-60 clients). Consequently, the 
first 18 to 24 months of the study were spent in adjunctive clinical and administrative balancing, while 
developing and practicing individualized programming as consistently as possible. 

The philosophy of individualization which FIAP sought to develop and implement implied rigorous 
practice of specific methods, consistent with its inherent values. Initially, one to one, office-based 
supervision of each FS by the FIAP clinical supervisor was provided using a mentoring model. This was 
augmented by consultation and weekly feedback and brainstorming sessions of the FIAP clinical team. 
However, what did not develop was an effective method of hand-on, field based oversight and 
supervision which may be necessary to enhance clinical accountability and program fidelity. 

The Family Specialist Log 

About 1 8 months into the program, a FIAP Family Specialist (Daily) Log Per Activity, per client, was 
developed and implemented for ten months&shyp;long enough to derive data to analyze what, how, 
when, where, how long and with whom FS activities were actually occurring. The log was a matrix for 
recording FS activities for each 24 hour period, including the following defined activities: Prepare 
Follow Up; Advocate; Facilitate or Coordinate; Inform or Communicate; Monitor or Supervise; Observe 
or Assess; Plan or Problem Solve; Counsel or Support; Provide or Broker; Reinforce; and Train. These 
activities were cross referenced with specific definitions for date, clock time, number of minutes, contact 
person(s), contact method, place of contact, and life domain(s) involved. An overall 65% inter-observer 
reliability was obtained across definitions, across Family Specialists. 



These data have been only partially analyzed as relates to the mean number of contacts, the total 
intervention time, and the types and times of contacts. Additional analyses are in process, especially as 
to relationships among program activities, children, FSs, and outcome measures, and as to variations in 
practice and effectiveness among the four FSs. 



Using data from these logs, client records, and clinical impression, 10 most and least successful FIAP 
clients were identified from the original 54. Demographic data and log data for these 20 children were 
analyzed, to begin to discriminate any critical aspects of FIAP's independent variable. Additional 
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analyses are in process, especially as to variations in practice and effectiveness among the four FSs. 
Return to fop 



Results 

Preliminary analysis indicated that the two groups of ten were homogenous as to ethnicity, gender, 
length of time spent in foster care, and mean total number of FS/client contacts. The two groups, 
however, were essentially different from each other in several ways, as shown in Table 1 . 

Return to fop 



Implications 

1. Individualized wraparound programs should begin at the earliest possible moment, at the earliest 
possible age. 

2. To the degree possible, staff support and supervision should be field based, perhaps, as opposed to 
mostly office or group based. 

3. Staff turn over is to be avoided, if at all possible. When it occurs, attempts to pair the incumbent 
with the new staff member should be made, for at least 2-3 weeks. This may enhance essential 
personal and professional familiarity with each child and family, as well as other key players. 

4. The differences in approaches and activities used by different Family Specialists need further 
analysis, however, this variable is probably the most difficult to measure. If it is assumed that 
intense, daily supervision of highly accountable, effective individualized wraparound services is 
required, personality profiles, prior experience, and specialized training for persons willing to 
work under such circumstances will be needed. 

Initially, FIAP thought that each child's service or treatment team would help ensure reliable, 
consistent Family Specialist activities once agreement was reached on a service 
plan&shyp;especially when paired with weekly FIAP team meetings and periodic one-to-one 
supervisory meetings. The variability of outcomes for children across FSs suggests that something 
more may be necessary to monitor and promote program fidelity, consistency and accountability. 
Unpredictable and essentially unmeasureable child, family, worker, system and program variables 
may require constant on-site monitoring and feedback&shyp;a very expensive and 
proposition&shyp;especially with professionals who believe they work best alone. 

5. The effectiveness of case managers can be maximized by matching case loads to the severity of 
the families' problems. Many "discharged" clients required rather constant checking and fine 
tuning&shyp;unfortunately, some families may never reach the independence to become their own 
successful case managers. FIAP found that 12 cases were often too many to handle effectively, 
and children who were "permanently placed" often required periodic, intensive follow along 
monitoring and supports. 
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Discussion 

The authors are encouraged by the positive results of the FIAP study, to date, including significantly 
better placement and permanency outcomes for FIAP children (Clark, Lee, & Prange, in press). We are 
confident that the recommended intervention practices that were developed will assist other 
professionals to more effectively establish individualized services for children and families lost in the 
foster care system (McDonald, Boyd, & Clark, in press). 
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The difficulties of accountable, consistent, reliable, faithful application of the programmatic philosophy, 
values, and enabling methods of highly individualized, wraparound services to foster children with 
severe emotional disturbance are varied and many. FIAP has learned that efforts to maintain, record, and 
reliably measure all aspects and activities of the application of its independent variable in the field are 
complicated and often confounded by the nature of individualized programming, itself. Yet, assuring the 
implementation of the essential elements of experimental independent (treatment) variables is critical to 
the validity of measures of experimental dependent (outcome) variables, regardless of research design. 

Though FIAP has been highly successful with certain of its clients, mean data analysis of the entire 
FIAP group may not discriminate either these successes, nor those specific factors within the 
independent variable most contributory to either success or failure. FIAP shall continue to try to ferret 
out those factors, through continued analyses of sub-group independent and dependent variables, in both 
experimental groups (e.g., FIAP and standard practice), but it remains unclear what to recommend to the 
field, as to how best to accomplish this in field-based research. 

The field of applied research in human services may need to recognize that far more (and sometimes far 
less) goes on in the real world of day to day treatment than projected, planned, or claimed by 
researchers, at times badly skewing results. Surely, this is even more true in highly individualized 
methodologies. Analysis of variance, and many other mean-based statistical analyses, do not allow for 
these inconsistencies, inadvertently resulting in Type I and Type II errors. 

It may be implausible to base scientific conclusions as to the efficacy of human service interventions and 
treatment paradigms on anything less than necessary and sufficient, accountable, reliable, accurate 
measures of treatment application, aggregated client by client. This being extremely difficult and 
unlikely, perhaps some sort of combination analyses of case study data, sub-group data (e.g., top/bottom 
groups), and total group mean data&shyp;involving reasonably reliable, multi-measured estimates of 
both types of variables&shyp;would be more effective. Perhaps FIAP can let you know. 

Keep tuned. 
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Introduction 

In recent years, child welfare agencies have been challenged with a drastic increase in referrals. For 
example, in 1992, 1.3 million reports were made on 2.7 million children nationally (Wells, 1994). These 
reports were made for a number of reasons, including neglect (45%), physical abuse (27%), sexual abuse 
(17%), and emotional abuse (7%) (McCurdy & Daro, 1994). In addition to the abuse and neglect, the 
children served by child welfare agencies often experience difficulties in school, struggle with substance 
abuse problems, and exhibit antisocial behaviors (Trupin, Tarico, Low, Jemelka, & McClellan, 1993). 

As reflected by the National Commission Report on Child Welfare and Family Intervention (1990), the 
social service agencies designed to help these multi-need children and youth often cannot provide 
adequate services to them. As a result, these children and youth may lose their ability to cope with a 
complex society, fail to successfully transition into adulthood, and drift into institutions and prisons. 

Kaleidoscope, Inc., Chicago, Illinois is an agency whose priority is to provide services to those children 
and youth that have not been effectively served by other social service agencies. These children have 
been diagnosed as seriously disabled, disturbed, or incorrigible and are in need of intensive treatment 
service efforts toward integrating them into their home communities and/or family life. The services 
Kaleidoscope provides are based on the philosophy of Wraparound Care which underscores the 
importance of unconditional care, intensive case management, individual planning, family involvement, 
flexible funding, and cultural competence. 

Kaleidoscope's Satellite Family Outreach Program uses these principles in serving their children and 
families that have experienced numerous difficulties and separations. In the present study, an extensive 
archival review was conducted on the files of all children and families served by Satellite during the 
years 1990 through 1993. The purpose of this study was to address the question. Who are the children 
and families served by the Satellite Family Outreach Program and what services did they receive prior to 
Satellite? 

Return to Top 



Method 

Setting 

This study was conducted at Kaleidoscope, Inc., a not-for-profit child welfare agency 
in Chicago, Illinois. In January 1994, Kaleidoscope received a grant from the Annie E. Casey 
Foundation to evaluate its Satellite Family Outreach Program. This study is one component of that 
overall evaluation. The Satellite program serves approximately 25 families at any one time and employs 
a staff of four administrators, three supervisors, four social workers, and 20 family workers. 
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Participants 

The families participating in this study were parents and children who had received services from 
Satellite during the years 1990 through 1993. Files were reviewed for 101 families that consented to 
participate. Information was collected on 133 parents/grandparents and 269 children. The parents and 
grandparents ranged in age from 19 years to 74 years old (M = 33.8 years). The racial composition of the 
adults was 71% African-American, 17% Caucasian, 8% Latino, and 4% Other. The majority of the 
parents were women (78%). The children ranged in age from 3 months to 20 years old (M = 8.93). The 
racial composition of the children consisted of 73% African-Americans, 12% Caucasians, 10% Latinos, 
and 5% Other. A slight majority of children were male (51%). 

Instrument 

The Demographic Overview Forml, Child Version (44 items) and Adult/Family Version 

(42 items) were designed to identify specific characteristics of Satellite clients. Specifically, the forms 

focused on background data, reason for referral to Satellite, presenting problems, and previous service 

utilization. 

Procedure 

Data collectors were oriented to file format and data collection methods in three steps. First, each data 
collector met with the first author to review and complete Demographic Overview Forms for a file. This 
was repeated until the data collector was knowledgeable about the file format and data collection 
procedures. Second, the data collector and the first author reviewed and completed forms independently. 
The completed forms were compared and any discrepancies were discussed. If the discrepancies were 
minimal, the data collector began to review files independently. Additionally, the first author reviewed 
20% of the files to assess reliability. Reliability ranged between 89% to 95% across the items with the 
average reliability being 93%. 
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Results 



Parents 

English was the primary language of 92% (N = 122) of this group. Many of the parents were single; 

37% (N = 49) were never married, 16.5% (N = 22) were divorced, 5% (N = 7) were widowed, 21% (N = 
28) were married, and 1 1% (N = 15) were separated. Most of the parents lived either alone with their 
children or with their spouse or significant other. A majority of the parents 
were not employed (63.2%). 

Prior to entering Satellite, the parents had received services from an average of 3.06 different agencies 
(see Table 3). The most frequently used agencies were the Department of Children and Family Services 
(96.9%), the Department of Public Aid (65.5%), private agencies (63.3%), and the Department of 
Mental Health (23.3%). Twenty-three percent ^ = 30) of the parents had been admitted to a medical 
hospital at least once, 14% (N = 18) had been admitted to a psychiatric hospital, and 10% (N = 1 1) had 
participated in an in-patient substance abuse treatment program. Many of the parents and their families 
lived at or below the poverty level (68.4%) and received assistance from the Department of Public Aid 
(65.5%). 



Children 

Forty-four percent of the children served by Satellite (N = 1 19) were in the guardianship of 
their mothers and 42% (N = 1 14) were in the guardianship of the Illinois Department of Children & 
Family Services. The guardianship of the remaining children was 5% (N = 13) both parents, 3% (N = 9) 
other relatives, 2.6% (N = 7) father, and 2.6% (N = 7) other. The children had been served by an average 
of 4.12 agencies prior to Satellite. Before entering Satellite, 75% of the children had been residentially 
placed outside of their natural home on at least 3 different occasions. A great number of the children 
placed (83%) had been in either relative, general, or private foster care. 

Families 

When a family was referred to the Satellite program, the referring agency indicated an average of 3.83 
reasons for referral. The top four reasons for referral were: the parent lacked the skills or resources to 
nurture her/his children (69.5%), the referring agency wanted to reunify the family members (47.5%), 
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the parent needed treatment for her/his behavior and emotional difficulties (46.8%), and the family unit 
had unmet mental health needs (45.3%). The permanency goals involved providing families services so 
that the children could remain in their home (55%), return to their natural home (42%), remain 
permanently in the home of a relative (12%), remain with a foster family (6%), or remain in a long-term 
care facility (1%). 
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Discussion 

The present findings indicate that the children and families served by an urban child welfare agency 
present many challenges to service providers. Many of the children live in single parent families that are 
financially supported through public aid. These parents struggle to provide for their children on incomes 
below the poverty level. A large percentage of families have been involved with state child protective 
services. Almost half of the children are wards of the state. It is important to note the lack of stability in 
these families' (especially the children's) lives. Many of the children had lived in multiple residential 
placements by the time they were 10 years old. 

The implications of the present findings are numerous. First, the information may be used to improve 
Satellite's services through an increased understanding of their clients' characteristics. Second, service 
utilization histories may provide a basis for Satellite staff to identify unique and creative resources their 
clients may need but are lacking. Finally, the findings may identify some characteristics of those 
children and families who seemingly fall through the cracks of existing public systems of care. These are 
the people who most need comprehensive, community-based services that will help them regain a stable 
family unit and link them to the resources found in their home areas. With those types of services in 
place, perhaps these families can become successful and independent. 
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Introduction 

A major consequence of the poverty, violence, and drugactivity in inner cities, such as in New York, 
Washington, DC and Baltimore, is the massive influx of children into the foster care system. On any 
given day, approximately 450,000 children nationwide are in foster care, presenting with more severe 
and complex problems than were seen a decade ago (Jost, 1991). 

The present study extends prior research on the prevalence of mental health problems among foster care 
children (e.g., McIntyre and Kessler, 1986). Such research is needed to develop effective treatment 
strategies for these "at-risk" children. 
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Method/Procedure 

Health Clinic personnel administer a combined medical and mental health screening protocol for all 
abused and neglected children entering out-of-home placement/foster care in Baltimore. A unique 
public-private partnership, the Health Clinic combines clinical staff from the Baltimore City Department 
of Social Services (BCDSS); Baltimore Medical System, Inc. (BMSI), a local non-profit community 
health organization; and Dale and Fogelman Associates, a group of psychologists and other mental 
health professionals who conduct the mental health screenings. Figure 1 displays the mental health 
screening model developed by Dale and Fogelman Associates. 

The following research findings are based on a random sample of 300 children from the 4073 who 
received mental health screenings at the Health Clinic in Baltimore between July 1, 1992 and June 30, 
1994. It was decided to select 100 subjects from each of three developmental age groups: 2 months to 5 
years, 8-12 years and 13-19 years (see Table 1 for a summary of the instruments and criteria used) . First, 
background information &shyp; such as race, sex and reasons for entering foster care &shyp; was 
tabulated in order to obtain a demographic profile of the population. Second, the cognitive and 
emotional functioning of these at-risk children was analyzed by developmental age group. 
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Results: Demographics and Mental Health Functioning 

Forty-nine percent of the selected sample were male and fifty-one percent were female. Ninety percent 
were African-American and nine percent White (see Figure 2 for a demographic summary). Chi-square 
analysis and analysis of variance were performed in order to determine whether either demographic 
variable&shyp;gender or race&shyp;had an effect on mental health functioning. No significant 
relationships were found between either demographic variable and scores on the Denver Developmental 
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Screening Test - Version II (DDST-II), Peabody Picture Vocabulary Test - Revised (PPVT-R-L), 
Developmental Test of Visual-Motor Integration (VMI) or Reynolds Depression Scales (RCDS and 
RADS). 

The most common reasons for entering foster care included maternal substance abuse (54%), neglect 
(53%), abandonment (27%) and physical abuse (22%), as shown in Figure 3. It should be noted that 
multiple reasons for referral could be listed. 

Fifty-one per cent of the children aged 2 months to 5 years were identified as in the "suspect for delay" 
range on the DDST II (see Figure 4T These findings are in stark contrast to normative data which 
suggest that 90% of children in this age group fall within the "normal" range (Frankenburg, Dodds, 
Archer, Bresnick, Maschka, Edelman & Shapiro, 1992). 

On the Reynolds Depression Scales (RCDS and RADS), 1 1% of the 8-12 year-old group and 30% of the 
13-19 year-old group reported significant depressive symptomatology (see Figure 5 ). The difference 
between age groups reached statistical significance [c2 (1, N=200)=l 1.08, p=.001j. The strength of the 
relationship between age group and depression was found to be moderate (r=.23). 

Whereas the rate for the older group is disproportionately high when compared to normative samples, 
the percentage of children in the 8-12 year-old group identified as evidencing significant depressive 
symptomatology is comparable to percentages found in normative samples (Reynolds, 1987; Reynolds, 
1989). 

The lack of a significant relationship between depressive symptomatology and gender for both age 
groups was surprising [c2 (1, N=100)= 1 .09, p = ns] in light of prior research documenting that girls tend 
to be more depressed than boys (Reynolds, 1987; Reynolds, 1989). 

Based on PPVT-R-L results, 53% of children in the 8-12 year-old group and 64% of children in the 
13-19 year-old group showed evidence of severe receptive language difficulties (see Figure 6). These 
figures are considerably higher than the 10% found in normative samples (Dunn & Dunn, 1981). No 
significant relationship was found between age group and scores on the PPVT [t(196)=.18, p= ns]. 

Both age groups demonstrated visual-motor integration skills that were closer to age expectation than 
were their receptive language skills (see Figure 6) . However, children aged 13-19 still evidenced 
disproportionately high deficit rates, with 37% scoring below a standard score of 80 on the 
VMI&shyp;as opposed to 10% in normative samples (Beery, 1989). For children aged 8-12, 13% of the 
foster care sample scored below 80&shyp;as opposed to 10% in normative samples (Beery, 1989). 
Children aged 13-19 scored obtained significantly lower standard scores (M= 84) than children aged 
8-12 (M=91.3)&shyp;[t(196)=4.20, p=.001]. 
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Conclusion/Policy Recommendations 

The present study describes the innovative mental health screening program implemented by Dale and 
Fogelman Associates in Baltimore. Preliminary research findings document the high rates of cognitive 
and emotional difficulties found among many of the children entering out-of-home placement/foster 
care. 

The following policy recommendations flow directly from the major policy innovations that the Health 
Clinic has implemented in Baltimore: 

1 . Centralized Screening Programs - In metropolitan areas, centralized clinics administered in 
conjunction with the local social service agency can ensure cost-efficient screenings. In 
suburban/rural areas, several identified medical practitioners may need to "share" one mental 
health practitioner who can perform screenings on-site for the purpose of generating consolidated 
health information for children entering foster care. 
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2. Consultation Programs - Mental health consultations should be conducted by clinic staff for social 
services workers. Such consultations can prove invaluable in clarifying recommendations and 
providing assistance in difficult cases. 

3. Computer Tracking - Both centralized and non-centralized programs can benefit from a common 
computerized tracking system. This database should include the following critical information on 
each child: placement, current health status, major recommendations (including need for 
follow-up), guardians, and the particular social service unit to which the child's case is being 
transferred. Such a database can also facilitate in-house record keeping regarding proficiency in 
foster care placement and follow-up. 

4. Provider Forums - Mental health providers, in conjunction with their medical counterparts, should 
confer with community practitioners who typically service children once they enter foster care. 
The overriding need for "continuity of care" suggests that provider forums can (1) help clarify the 
policies of the screening clinic; (2) smooth transitions for children as they move in and out of the 
foster care system; (3) arrange for the pooling of data; and (4) "bond" clinicians in the common 
cause to serve at-risk children. In non-urban areas, bi-annual meetings may be preferred, 
especially if local practitioners know each other well. The importance of forums designed to 
oversee the delivery of services to children in foster care should not be underestimated. 

5. Program Linkages - Mental health providers need to forge liaisons with Head Start, pediatric HIV 
programs, and other public and private child advocacy groups. Program linkages are especially 
important in removing bureaucratic barriers that impede service delivery to children. 
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Introduction 

The complexity of the emotional and behavioral problems youth experience thwart efforts to develop 
integrated and coordinated systems of care for children and adolescents (Duchnowski, Johnson, Hall, 
Kutash, & Friedman, 1993). The difficulties of helping youth with multiple problems may be 
compounded when the problem set includes behaviors that span the mental health system and the 
juvenile justice system, specifically. Studies of the general population of adolescents show delinquency 
and mental disorder co-occur (e.g., Elliott, Huizinga, & Menard, 1989). Few studies, though, have 
estimated the proportion of youth referred to juvenile courts in need of mental health services or the 
proportion of mentally disordered juvenile offenders in mental health placements (Fagan, 1991). This 
summary explores (1) the prevalence of delinquency among youth referred for services within the 
mental health system, (2) the co-occurrence of delinquency and mental disorder, and (3) the prevalence 
of official delinquency as indicated by involvement with the juvenile justice system. 
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Method 



Sample 

Data were obtained through the Fort Bragg Evaluation Project (FBEP), an evaluation of a model mental 
health service system for children and adolescents (see Bickman, Guthrie, Foster, Lambert, Summerfelt, 
Breda, & Heflinger, 1995, for more details on the evaluation and the model service system). The sample 
included children and adolescents ages five through seventeen who were referred for formal mental 
health services at the time they entered the study. On average, youth were about 1 1 years old, and about 
60% were male. Seventy percent were white; about 20% African American; and 10% were of other or 
mixed race/ethnicity. Less than half of the sample lived with both biological parents, though 80% lived 
in a two-parent household (e.g., with a biological and stepparent). Most of the families earned between 
twenty and forty thousand dollars annually. The sample's average standardized scores on the Child 
Behavior Checklist (CBCL) and Youth Self Report (YSR) were very similar to those reported by 
Achenbach (1991) and Achenbach and Edelbrock (1993) for other clinical samples of youth. Thus, 
findings from this research may apply to other populations of roughly middle-class youth referred for 
mental health services (see Bickman, et al., 1995, for more details on the representativeness of the 
sample). 

Measures 

Delinquency. The CBCL's (Achenbach, 1991) standardized narrow-band scale score on the delinquency 
syndrome measures level of delinquency-related problems 1. Cutoff points for the scale also indicate 
whether youth fell into the borderline/clinical range (hereafter, clinical), or the nonclinical range. 
Official delinquency refers to contact with the juvenile justice system; that is, whether youth either had 
been picked up or arrested by police, punished or adjudicated by a court, placed on probation, or 
incarcerated. 
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Mental Disorder/Psychopathology. Seven standardized CBCL scales-p;withdrawn, somatic complaints, 
anxious/depressed, social problems, thought problems, attention problems, and sexual 
problems-p;measure level of psychopathology2. Again, cutoff points for each of the seven scales 
indicate whether youth fell into the clinical or nonclinical range for each problem area. An overall 
psychopathology measure identifies youth who fell into the clinical range for any of the seven 
syndromes. 
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Results 

Delinquency 

Table 1 shows the sample's average T-score on the delinquency syndrome was 64.4. Forty-seven percent 
of the youth fell into the clinical range. Nearly one-fifth (18%) of the youth had contact with the juvenile 
justice system, a rate roughly six times that found in the general population (U.S. Department of Justice, 
1991). As expected, age and delinquency positively correlated, with regard to delinquent 
symptomatology and official delinquency. Particularly, delinquency and legal contact increased 
dramatically among older adolescents 12 to 17 years of age. Males also were significantly more likely 
than females to have had contact with the police or courts. Contrary to the delinquency literature (e.g., 
Elliott, et al., 1989), however, they showed lower levels of delinquency-related problems than females. 
African Americans, too, showed lower levels of delinquency than either Whites or youth of other races. 
These findings may reflect a selection bias, in that more seriously delinquent males and African 
Americans may be less likely to receive services within the mental health system than to be referred to 
the juvenile justice system, compared to females and White youth. 
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Mental Disorders 

Table 2 shows the proportion of youth in the clinical range on each of the narrow-band syndromes. 
Considering the seven measures of psychopathology, 70% of the youth experienced clinical levels in at 
least one area. Forty-one percent of youth manifested clinically-significant levels of aggressiveness. 
However, more youth reported serious delinquency (47%) than any other type of problem. All nine 
syndromes considered, 80% of the sample reported clinical levels in at least one problem area 3. 
Co-occurrence of Delinquency and Mental Illness 

Table 3 presents the zero-order correlations between the delinquency scale score and the seven 
psychopathology scale scores. The data show higher levels of delinquency associated with higher levels 
of psychopathology. While all the relationships are statistically significant (p < .01), the magnitude of 
the correlations is rather modest. Nearly all are .4 or less. In general, these findings suggest a moderate 
relationship between levels of delinquency and psychopathology. 

Another way to assess co-occurrence of delinquency and mental disorder is to identify the proportion of 
youth with clinical levels of symptomatology in both areas. Table 4 shows that 39% of the youth 
experienced clinical levels of delinquency-related problems as well as clinical levels in at least one of 
the seven areas of psychopathology. At the same time, a nearly equal proportion (31%) had more 
focused problems of the type mental health services were traditionally designed to address. 

A closer view of these data further highlights the nature of the relationship between delinquency and 
mental disorder. Among youth with serious levels of psychopathology (70%), 56% also indicated a 
clinically-significant level of delinquency. The problems of the other 44% remained more circumscribed 
to some type of mental disorder only. The picture changes considerably, though, when one considers 
those youth referred for services with serious delinquency-related problems (47%). Among these youth, 
a large majority (83%) also had clinically-significant psychopathology. The pervasiveness of these 
youth's problems is ftirther indicated by the fact that youth with clinically-significant delinquency 
problems reported clinical levels in an average of 3 .3 areas of psychopathology while those with a 
nonclinical delinquency profile experienced serious levels in 1 .6 areas of psychopathology. So, while 
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delinquency and mental disorder are only moderately related, when delinquency occurs, it is very likely 
to be accompanied by severe psychopathology in multiple problem areas. 
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Discussion 

The co-occurrence of mental illness and delinquency, specifically, raises special concerns for the mental 
health field. The large and growing numbers of comorbid youthfol offenders within the mental health 
system, and the pervasiveness of their prob-lems, challenge the way we develop, structure, and deliver 
services to this group of youth. Programs within the mental health system have been traditionally 
designed for troubled youth, not troublesome youth. The findings presented here suggest that clinicians 
may need to anticipate that over half of their clients may not only have some psychopathology, but may 
also be disruptive, perhaps violent, and difficult to control. 

The task of providing appropriate care is made more difficult by the fact that relatively large proportions 
of youth in the mental health system have been involved in the juvenile justice system as well. 
Developing a service system for troubled and troubling youth requires integration of more than the 
typical child-helping systems (e.g., education, social services). It also requires incorporating the juvenile 
justice system, a system that unlike other child-serving systems, has increasingly responded punitively 
rather than rehabilitatively toward youth (Schwartz, 1989). Problems of system integration and the 
development of effective services extend beyond these philosophical differences between systems. 
Pragmatic issues of system boundaries, perceived scope of responsibility, and financing structures 
continue to thwart efforts to integrate service systems for children and adolescents (Duchnowski, 
Johnson, Hall, Kutash, & Friedman, 1993; Fagan, 1991). 

The overlap between mental disorder and delinquency, indicated by the findings here, also calls into 
question the criteria for placing youth in either the mental health or juvenile justice system. Some 
evidence suggests, for example, that personal traits such as sex and race predict system placement more 
reliably than psychopathology (Westendorp, Brink, Robertson, & Ortiz, 1986). The juvenile justice 
system has long been prey to charges of discrimination. The mental health system, with some historical 
exceptions, has been less a target for such claims. This situation may change as the types of problems 
that define youth in need of mental health treatment and eligibility for reimbursement expand, such as in 
the case of substance use (Schwartz, Jackson-Beeck, & Anderson, 1984; Staples and Warren, 1988). 

The development of a service system that can adequately attend to the needs of youth who demonstrate 
levels of delinquency-related problems more severe than 95% of their peers in the general population 
remains a serious challenge for policy makers, program planners, and clinicians. Moreover, we must also 
recognize that the juvenile justice system must be counted among the institutional actors that represent a 
comprehensive system of care for children and adolescents (Stroul and Friedman, 1986). Efforts to 
develop this integrated system, finally, cannot be divorced from larger issues of social justice and equity. 
Return to Top 
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1 The delinquency syndrome scale is based on 1 3 items, 7 of which refer to legally proscribed behavior 
(e.g., nxnning away, arson, stealing at home, stealing outside the home, truancy, substance use, and 
vandalism). Three other items refer to nondelinquent behavior (swears, lies/cheats, associates with 
troublemakers). Three others refer to attitudes or thought processes (feels no guilt after wrongdoing, 
prefers older companions, and thinks about sex too much). Thus, the delinquency measure used in this 
research is not based solely on delinquent conduct. However, empirical analyses have repeatedly found 
that these thirteen items correlate and load on a single "delinquency" factor (Achenbach, 1991). Further, 
preliminary analyses of the FBEP data show that over 80% of youth with clinical levels of delinquency 
also report engaging in legally proscribed behavior. These findings support the validity of using the 
delinquency syndrome score as a surrogate for delinquency. 

2 For this research, the author does not consider the aggressiveness syndrome a measure of 
psychopathology. It more resembles delinquency: it includes three items that refer to legally proscribed 
behavior (i.e., destroying others' property, fighting, and physically attacking other people) and highly 
correlates with delinquency (r = .71). 

3 That 20% did not report a clinical level on any of the CBCL measures does not necessarily mean some 
youth were receiving mental health services inappropriately (Achenbach, 1991). 
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Introduction/Purpose 

The purpose of the present study is to describe the mental health functioning of youth involved in the 
mental health and juvenile justice systems. Self-report information is derived from the Millon 
Adolescent Clinical Inventory (MACI; Millon, 1993), which gives diagnostic impressions and 
personality descriptions, as well as from the Trauma Symptom Checklist for Children (TSCC; Briere, 
1989), which gives information about trauma related symptoms and post-trauma functioning. Caregivers 
report impressions of behavior and emotional functioning using the Child Behavior Checklist (CBCL ; 
Achenbach, 1991). 
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Method/Procedure 

Two self-report instruments&shyp;the MACI and TSCC&shyp; were administered to 13 male 
adolescents incarcerated in a juvenile justice facility (Group I) and 13 male adolescents served by 
intensive-in-home wrap-around services (Group II). The CBCL was completed by caregivers for both 
groups. 

There were no statistically significant differences in age or race between the two groups. The mean age 
for the two groups was 15.5 years for Group I and 14.5 years for Group II (t = 0.23, df = 24, p = NS) 
with a total range of 13 to 17 years. The racial distribution was: 3 African-American, 10 Caucasian for 
Group I and 1 African- American, 12 Caucasian for Group II (c2 > 0.295, p > .05). 

Additionally, a demographic coding sheet was completed on all subjects by the investigators. 
Demographic data was obtained from juvenile justice and mental health databases which included child, 
family and academic information. Youth were comparable on age, race, gender, and on mental health as 
well as juvenile justice issues. 
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Results 

Results on the MACI suggest that youth in the juvenile justice system expressed significantly more 
mental health concerns than did the youth in the mental health system. Overall, youth in Group I 
displayed elevations on 1 1 scales while those in Group II were elevated on 8 scales. Five areas of 
significant differences between the two groups were found. Significant differences were in the direction 
of Group I. (see Figure 1) Significantly elevated scores on MACI scales included Doleful, Desirability, 
Childhood Abuse, Substance Abuse Proneness and Anxious Feeling. 
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It is also important to note that no differences were found between the groups on MACI scales 
measuring Depressive Affect, Suicidality or Delinquent Predisposition. Also, groups were comparable 
on the TSCC scales measuring Anxiety, Depression, Post-traumatic Stress, Sexual Concerns, 
Dissociation and Anger. 

Many significant differences were found between the caregivers' ratings on the CBCL for the two 
groups. However, the differences are difficult to interpret. The validity of observations reported for 
Group I are questionable. Observations for Group II showed a strong relationship to youth-reported 
concerns on the MACI. 
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Implications 

The present study identifies the mental health concerns of a small number of youth involved in the 
mental health and juvenile justice systems. The study directly measures the concerns of youth in each 
system. Differences as well as similarities describe mental health concerns in the juvenile justice 
involved youth. Understanding the breadth of the juvenile justice involved youth's mental health needs 
may facilitate the development of an array of services addressing the physical, emotional, social and 
educational needs of the adolescent and his/her family. 

Two major shortcomings of the study are its small sample size and possible group difference resulting 
from subject selection procedures. Youth in the juvenile justice group resided in an institution, while 
youth in the mental health group resided at home and were served by community-based interventions. 
The mental health group is the most comparable group available in the public mental health system in 
Ohio, but group differences resulting from the effect of institutionalization may account for the study's 
findings. Further cross-system work is needed. 
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Background 

The problems of juvenile crime and the incarceration of youth are receiving increasing attention 
nationally. These are particularly critical problems in South Carolina, which already has the highest rate 
of incarceration of any region in the free world (Corrections Compendium, 1991; Fowler, 1991). The 
South Carolina Department of Juvenile Justice (DJJ) has a daily census of 800 to 1000 youth 
incarcerated in its facilities in Columbia and serves over 3000 youth annually. It is currently facing a 
lawsuit in federal district court involving overcrowding of its facilities and the care of juvenile offenders 
that have emotional disturbances or mental illness. 

The fact that youth with mental illness are being incarcerated at all brings up serious concerns about the 
failure of community services designed to prevent such outcomes (Duchnowski, Johnson, Hall, Kutash, 
& Friedman, 1993). The lack of community-based and preventive mental health services for youth, 
particularly in rural areas, allows many youth with emotional or mental disorders to go undetected and 
untreated to the point that they suffer from disruptive behavioral symptoms and come to the attention of 
juvenile authorities and family courts. These factors may account for the over-representation of 
African-American youth in the incarcerated juvenile population (Cohen, Parmelee, Irwin, Weisz, 
Howard, Purcell, & Best, 1990). There may be a number of factors involved in the over-representation 
of minority youth in incarcerated populations including: the failure of mental health and social services 
to address the needs of minority youth (Mason & Gibbs, 1992) and biases in assessment (Fabrega, 
Ulrich, & Mezzich, 1993) which may predispose the entry of minority youth into the juvenile 
correctional system. 

To date, there have been no systematic studies in the U. S. addressing the problem of incarceration of 
youth with emotional disorders/mental illness. The few studies conducted have not used systematic 
diagnostic instruments to evaluate such disorders and have not examined factors leading to incarceration. 
Those conducted so far indicate that a major proportion of incarcerated juvenile offenders have 
significant mental/emotional disorders (Hollender & Turner, 1985; McManus, Alessi, Grapentine, & 
Brickman, 1984). 

Specific Aims 

This project addresses a number of important questions underlying the incarceration of youth with 
mental/emotional disorders. These include: (1) What is the prevalence of major DSM III-R diagnoses 
amongst incarcerated versus hospitalized youth? (2) What is the symptomatic prevalence (mean 
symptom count) across the two groups? (3) What are the levels of behavioral symptomatology (as 
measured by CBCL and YSR) across the two study groups? 
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Subjects 

We recruited youth from the midlands region of South Carolina, ages 13-17, including youth 
hospitalized at the William S. Hall Psychiatric Institute (which serves youth from most of the state) and 
incarcerated youth recruited from DJJ in Columbia. Hospitalized youth were randomly selected from 
new admissions, while incarcerated youth were randomly selected from institutional rosters. The 
counties included in the study represent a balance of rural and urban populations. Youth with an IQ of 
65 or below were excluded from the study. Sample demographics were as follows: mean age = 14.9 
(hosp.), 14.4 (DJJ); gender = 55% male (hosp.), 94% male (DJJ); 
race = 52% African-American and 48% Caucasian (hosp.), 

68% African-American and 32% Caucasian (DJJ). 

Instruments 

DISC-PC, version 2.3 (Diagnostic Interview Schedule for Children). This structured diagnostic 
interview was used to determine diagnoses as well as symptoms of mental/emotional disorders. It was 
administered by psychiatric social workers, nurses and physicians (Shaffer, et ah, 1993). 

Child Behavior Checklist. This instrument was used to obtain parent ratings of behavioral and emotional 
symptoms (Achenbach, 1991). 

Services History Form. The Services History Form was developed and administered as a standard 
instrument, completed by the parent to document and code categories of health, mental health (outpatient 
and acute hospitalization), residential, educational, social, and volunteer (Boy/Girl Scouts, Big 
Brother/Sister, etc.) services provided for each youth. This instrument was patterned after that described 
by Hinkle (1992), in use at the South Carolina Continuum of Care. Parents were asked to rate the value 
of this service, the age of the youth when the service was used, the service's duration, and frequency. 

Procedure 

Youth and their parents/guardians were recruited to participate in the study and written consents were 
obtained. The parent behavior rating scale (CBCL) was completed either in person or by mail. If the 
parents were unable to read they were provided assistance. The youth were then interviewed by trained 
professionals (clinical psychiatric social workers and nurses). 
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Results 



We are currently reporting on preliminary findings based on 3 1 hospitalized and 3 1 incarcerated youth 
studied to date. On the DISC 2.3, we found that 79% of the hospitalized youth and 71% of the 
incarcerated youth met diagnostic criteria for at least one disorder. There was a high rate of comorbidity 
in both samples, with a mean of 3. 1 diagnoses per youth in the hospital sample and 2.4 diagnoses in the 
incarcerated sample&shyp;these differences in rates between the two samples were not statistically 
significant (t = 0.93, p > 0.20). Levels of symptomatology were also quite high in both samples, with a 
mean of 43.8 positive symptoms in the hospital sample and 30.4 symptoms in the incarcerated 
sample&shyp;a difference of marginal significance (t = 1.95, p < . 10). 



Using the CBCL, we found greater differences between the two groups. The mean total T score (72.8), 
the mean internalizing T score (70.4), and the mean externalizing T score for the hospital sample (70.0) 
were higher than for the incarcerated sample (mean T total = 61.8; mean internal T = 56.8, mean external 
T = 63.2; total T: t = 4.62, p <. 001; internal T: t = 4.84, p < .001; external T: t = 2.77, p < .01). In the 
hospital sample, 93% met the 60+ outpoint total problem T score suggested by Achenbach (1991). In the 
incarcerated group, 60% met the 60+ outpoint. Additional analyses along racial/ethnic lines indicate that 
hospitalized African-American youth (N = 15) had much higher mean number of diagnoses (3.56), mean 
symptom counts (50.8) and percent caseness (88%) than Caucasian youth who were hospitalized (N = 

15, mean dx. = 2.46, mean sx. ct. = 35.3, and percent caseness = 69%); while incarcerated 
African-American youth had slightly lower levels of psychopathology (N = 21, mean dx. = 2.21, mean 
sx. = 29.2, percent caseness = 69%). 



The overall levels of service utilization were foimd to be somewhat higher in hospitalized (mean = 7.90) 
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versus incarcerated youth (mean = 5.55; t = 1.76, p < .20). Levels of educational and volunteer services 
were similar (hospitalized youth: mean educational services = 1.63, mean volunteer services = 0.90; 
Incarcerated youth: mean educational services = 1 .32, t = 1 .49, p < .20, mean volunteer services = 0.7 1 , t 
= 0.60, p > .20&shyp;non significant). Incarcerated youth actually had higher levels of previous 
out-of-home residential services (residential, foster care, group home; 

mean = 1 .32) as compared to hospitalized youth (mean = 0.71 ; t = 2.22, p < .05). However, they had 
lower levels of previous mental health services (mean = 1.19) as compared to hospitalized youth (mean 
= 2.46; t = 3.05, p < .01). In examining cross-ethnic differences, we found that African-American 
hospitalized youth used equal levels of mental health services as Caucasian youth, but had higher levels 
of special school services (53% using additional school services) as opposed to Caucasian hospitalized 
youth (20%), higher levels of residential services (mean = 0.87) than hospitalized Caucasian (mean = 
0.47), but equal residential services as incarcerated African-American youth (mean = 1 .00). 
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Conclusions 

Levels of psychopathology and symptomatology are lower in incarcerated youth as compared to 
hospitalized youth. However, their levels of symptomatology and psychopathology are quite significant 
as compared to expected general population rates. This strongly indicates the appropriateness of high 
levels of mental health services to address the needs of incarcerated youth. In addition, the findings do 
not support the common assumption that incarcerated youth receive significantly fewer services than 
other youth. However, it is important to note that incarcerated youth have received lower levels of 
mental health services while receiving higher levels of out-of-home services. A tentative hypothesis 
coming out of our findings is that more frequent use of outpatient mental health services, perhaps at a 
younger age, might result in youth being served more readily in the mental health versus the juvenile 
justice system. Another tentative hypothesis is that minority youth may travel a parallel service pathway 
from school or social services into the juvenile justice system which does not often intersect with mental 
health services. Continued analysis on the complete cohort will investigate racial/ethnic factors in 
service utilization and age of entry into the mental health system across the two groups. 
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Background 

The Pinellas Boot Camp Initiative in Pinellas County, Florida, is a complex juvenile justice program 
characterized by a high degree of interagency collaboration, strong involvement of community leaders 
and policy makers, and an emphasis on family involvement. It is the first program of its kind in Pinellas 
County and one of the first programs in the country to utilize the boot camp model for juvenile 
offenders. 

The interagency service delivery model was designed to address concerns about increases in juvenile 
crime and recidivism with innovative solutions. Partners in this effort include the Pinellas County 
Sheriffs Department, the Pinellas County School Board, District V Juvenile Justice Program (JJ), 

Eckerd Family Youth Alternatives (EFYA), and the Juvenile Welfare Board of Pinellas County (JWB). 
The Advisory Group includes leadership from the County Commission, Office of Justice Coordination, 
and the Judicial system. The Juvenile Welfare Board's Research & Development Center is conducting a 
comprehensive evaluation of the Initiative to determine the profiles of the youth for whom this program 
is effective in terms of recidivism, their personal growth and public safety. 

The Pinellas County Boot Camp Initiative serves juvenile offenders (ages 14-18) and their families. 
Screening criteria were established by the Juvenile Justice Act. Eligible youth are those committed to 
level 6 and level 8 programs who are physically and mentally able to participate. The program includes a 
four month residential commitment phase (Pinellas County Boot Camp), where youth confined to the 
Boot Camp undergo intensive physical training, education and counseling; an eight month intensive 
aftercare commitment phase (Eckerd Leadership Program), where youth continue activities which now 
include restitution, career development, job training and employment in an increasingly less structured 
setting; and twelve months of voluntary follow-up. Family support services are provided to families of 
recruits during the entire Initiative. 

The Pinellas County Boot Camp serves the Juvenile Justice West commitment service region, which 
includes Pinellas, Pasco, Hillsborough, Polk, Highlands, Hardy, Manatee, Sarasota, Lee and Collier 
Counties. The Eckerd Leadership Program (ELP) serves only youth and families living in Pinellas 
County. Recruits from other counties are transferred to aftercare programs in their county of origin. 
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Method 

A comprehensive interagency data base and data management agreement were developed during the 
start-up year. The data base will be utilized in 1995-96 to track program outcomes and look for 
predictive factors, such as individual youth characteristics or patterns of offense history, which may be 
associated with successful program completion and positive outcomes. Other assessments have included 
personal interviews with recruits and staff at both facilities, timed observations, and regular data 
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collection from all collaborators about the youth and their progress 
in the Initiative. 

The creation of the new Florida Department of Juvenile Justice and associated changes in legislation and 
funding will have long lasting impact on many aspects of the Boot Camp Initiative. The most significant 
short-term impact was a delay in funding for full implementation of the Aftercare phase. 

Funding for the residential program at the Boot Camp was not affected when responsibility was 
transferred to the new Department on July 1, 1994. In addition, the Boot Camp, which opened in 
November, 1993, was past the start-up period with most planned interventions in place when the change 
occurred. By contrast, the ELP Aftercare program, which opened on March 29, 1 994 with start up funds 
from the Juvenile Welfare Board, did not have an established contract with the state's Department of 
Health and Rehabilitative Services to transfer to the Department of Juvenile Justice on July 1, 1994. 
Although the day program and some support services were provided, evening and weekend hours and 
the hiring of additional staff at the Aftercare site were delayed through the Fall of 1994. This resulted in 
less intense supervision of youth during the community phase than had been planned, and delayed 
implementation of some Aftercare programs and services. Full funding has now been approved, and the 
Aftercare component is moving towards full implementation. 

The collaborative program model is designed to provide comprehensive services to respond 
to complex needs. Program strategies include provision of structure and discipline combined with 
appropriate role modeling, individualized planning, supervision and family involvement to increase 
youths' pro-social behavior and reduce delinquency. Interagency communication and coordination are 
viewed by all participating agencies as essential to successful outcomes. Progress has been made to 
establish continuity and commonalties among all program components. 

An interagency planning and evaluation process has supported this effort. Continued attention to 
integration of program components is recommended, as is periodic interagency review of the conceptual 
framework and program goals. 
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Findings 

Data collected during the first year of the program show that youth admitted to the Boot Camp have 
diverse needs. Many have experienced disruptions in family life and have had poor school adjustment. 
More than half of all recruits have special education needs. Many have had several prior referrals into 
the juvenile justice system and have been in other commitment programs. Appropriate home placement 
during aftercare has been an issue for several youth, especially those who had been living in foster care 
or other non-family settings prior to commitment. Comprehensive evaluation of individual risks and 
needs has not always been available prior to placement in the residential Boot Camp program. 

Program completion data are available for all recruits admitted to Pinellas County Boot Camp, and for 
Pinellas County youth who transitioned to ELP Aftercare. Completion data is not available for youth 
who reside in other counties but will be included in the next semi-annual report. Recidivism rates and 
other outcome data were not computed because some phases of the program have been in operation for 
less than a year. Preliminary outcome data will be included in the semi-annual report. Because of 
changes in program eligibility, program modifications during the start up period and unexpected delays 
in implementing some aspects of the program, results for the first several platoons will be analyzed 
separately. 
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Discussion 

Other evaluation issues to be addressed during the next period include utilization and impact of 
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"recycling" youth who engage in risk taking behavior back to Boot Camp from aftercare, criteria for 
"readiness" of youth to transition from Boot Camp to a community setting and analysis of educational 
progress and outcomes during all program phases. 

Results to date are inconclusive. However, it is the expectation of the researchers that the next report on 
this Initiative will have preliminary information about the profiles of youth who most benefit from this 
program. 
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Introduction 

Within the past two decades, several trends have been instrumental in affecting the policy and 
intervention for families with children who have severe emotional and behavioral problems. In 
particular, the trends towards deinstitutionalization, family preservation, family reunification, and 
parent-professional collaboration have been synthesized to provide a coordinated systems policy to 
guide state mental health practice (e.g., Behar, 1988; Burchard, et al., 1991; Homonoff & Maltz, 1991). 
One key aspect of the emerging systems of care is the creation of family strengths and community based 
intensive child case management services. In an effort to increase our understanding of the factors 
associated with the successful implementation of family strengths based intensive case management 
services, this summary will focus on the results of an exploratory study, conducted to describe the efforts 
of one rural state's implementation of a family-strengths, community based intensive case management 
program (ICM) for families with children with severe emotional and behavioral problems. 
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Methods 

An in-depth, systematic review of ICM case records was undertaken for the purpose of: (a) delineating 
characteristics of the children and families being served; (b) identifying service needs and barriers; (c) 
describing the amount and type of contact between case managers and families and other service 
providers; and (d) examining the relationship between case managers' contact with families and 
providers, and characteristics of the families and children being served. 

Program Description 

This study was conducted following the first year of implementation of a state wide intensive child case 
management services. The intensive child case management services are part of a continuum of child 
and adolescent mental health services developed and implemented by the State of Maine Department of 
Mental Health and Mental Retardation. This continuum of services (e.g. child case management, crisis 
services, community-based residential services, etc.) was legally mandated following agreement of a 
class action suit between mental health consumer groups and state mental health and human services 
officials, resulting in a substantial decrease in the number of state hospital beds for adults, a virtual 
elimination of state hospital beds for children and adolescents, and increased funding allocations for 
improving and expanding community based mental health services. 

Six agencies were awarded two year contracts with the state department of mental health to provide 
intensive child case management services. Three of the agencies are community mental health centers, 
and three are private, non-profit child and family services agencies. There are a total of 1 3 child case 
managers employed throughout these six agencies, with each case manager expected to carry a caseload 
of 15 families with children with severe emotional and behavioral disabilities. 
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Population and Sample 

The population consisted of the case records of all Maine children and adolescents who received 
Intensive Case Management (ICM) services from the initial start of the program in April, 1991 through 
June, 1992 (approximately 150 children). Based on discussions with state department of mental health 
staff, case managers and agency administrators, and the review of the literature on children with severe 
emotional and behavioral difficulties [e.g., Lourie, et al (1990); Ronnau, (1992); Stroul & Friedman, 
(1986)], a purposive sample of 50 case records was chosen for review in order to compare across several 
categories of records: children involved in a state class action consent decree (20), children involved in 
the class action consent decree who were also in the custody of the department of human services (5), 
children who were psychiatrically hospitalized while receiving ICM services (10), children who 
remained in the community (without a more restrictive placement) while receiving ICM services (6), and 
children who were under the age of 12 when they began receiving ICM services (5). Each ICM Program 
supervisor randomly chose five cases for review, one case for each of the five categories. Sixty-eight 
percent (34) of case records involved boys; 32% (16) involved girls. The mean age for the children and 
youth was 13.42 years (SD: 2.88 years), with a range from 4 to 17 years. Of the 41 records noting the 
child's grade level, the mean grade level was 7.52 (SD: 2.61). The majority of the children and youth 
were White (88%, n=44), with 6% (3) Native American children and 6% (3) African American children. 

Procedures and Data Analysis 

Case records were reviewed by either the principle investigator or one of two graduate social work 
research assistants who had been trained by the PI. Three protocols were used in the review of the case 
records. First, the Child Case Management Case Record Review Protocol (developed by the principle 
investigator) consisted of a series of 1 8 close-ended questions related to child and family demographics, 
referral sources, family structure, and child's place of residence, and 27 open-ended questions on topics 
such as service needs, family background, child problems, and family strengths. Second, a revised 
version of the Vermont Restrictiveness of Living Scale (Burchard et al., 1991), was also completed for 
each case review, creating a graph of the number and type of placements experienced by the child from 
birth to the present review. Third, the Child Case Manager Contacts Protocol (developed by the principle 
investigator) provided a mechanism for recording case managers amount of contact (in hours), with 
whom (e.g., child, family, child therapist etc.), and the type of contact noted in each case record (direct, 
telephone, meetings, other). Each Child Case Management Case Record Review Protocol was read by 
the principle investigator and the two research assistants in order to look for emergent themes. Content 
analysis, based on the frequency with which themes were present in the protocol, was then conducted. 
Return to Top 



Results 

Child Characteristics 

At the time of referral to ICM 46% (23) of the children and youth resided with their families, 14% (7) 
were psychiatrically hospitalized, 1 2% (6) were in foster care, 1 2% (6) resided in an in-state residential 
treatment center (RTC), 10% (5) were at the Juvenile Detention Facility, 2% (1) lived in a therapeutic 
foster home, out of state RTC (2%, n=l), or a friend's house (2%, n=l). Birth parents were 
overwhelmingly the guardian of the children and youth at the time of referral (70%, n=35), followed by 
the department of human services (20%, n=10), and the child's adoptive parents (10%, n=5). 

Families were most frequently referred for ICM by the state department of mental health (32%, n=16), 
department of human services (18%, n=9), psychiatric hospitals (16%, n=8), and local crisis intervention 
services (8%, n=4). Families waited a mean of 9.60 months (SD: 4.72 months; range: 0-18 months) from 
the date of referral for ICM services to the actual introduction to a case manager. 

Table 1 shows the percentage of child presenting problems noted in the records at case opening. Case 
records noted a total mean of 3.06 (SD = 1.01) presenting problems at the time cases are opened. 

Family Characteristics 
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Information pertaining to the child's mother was found in 90% (45) of the case records. The mean age 
for the child's mother was 37.35 years (SD: 5.91, Range: 29-50). Fifty-eight percent (29) of the case 
records noted information about the child's father, with the mean age for fathers 40.21 years (SD: 6.23, 
Range: 29-49 years). The mean number of siblings (birth and step siblings combined) per family was 
2.15 (SD: 5.68, Range: 0-16). Family problems were noted in all fifty of the case records ( see Table 2), 
with a mean number of 2.42 family problems recorded (SD: .91, Range: 1-4). 

Ninety-two percent (46) of the case records made mention of family strengths with the predominant 
strength noted as the family is "interested and supportive," followed by "mother is a strong advocate for 
the child" (32%, n=16), the extended family is supportive (22%, n=l 1), and characteristics of the child 
such as talented, gifted or insightful (20%, n=10). The mean total number of family strengths noted in 
the records was 1.39 (SD: .58; Range: 1-3). Only 48% (24) of the records mentioned the child and 
family's vision and/or aspirations for the fiiture. Common visions for the future included the child 
"living a normal life" (46%, n=23), the child getting a good education or vocational training (36%, 
n=18), and the child getting along well with others (24%, n=12), . 

Service Characteristics 

The majority of families received out-patient mental health services (68%, n=34), and one-third of the 
families (15) received school related services. At the time of referral for case management services, the 
referral sources recommended the need for family support services in 74% (37) of the case records, 
followed by case management/coordination (64%, n=32). There continues to be a gap between services 
requested at the time of the Individualized Service Plan (ISP), and services obtained (at the time of the 
case record review). For example, as Table 3 indicates, 78% (39) of the ISP's recommended mental 
health services for the child with 54% (27) of these services actually obtained at the time of case record 
review. Educational services was another frequent item on the ISP (70%,n=35), with 50% (25) of 
families obtaining educational services at the time of case record review. 

The most frequently cited barrier to services and resources was the "lack of specific services" (38%, 
n=19). Other barriers to services cited in case records included problems arising from the child's 
behavior (38%, n=19; e.g. the difficulty in finding a therapeutic foster home for a child who sexually 
acts outO, inter-agency issues (16%, n=8), funding problems (12%, n=6), problems arising from family 
issues (12%, n=6), and cultural issues (4%, n=2). 

Child Placement History 

Table 4 provides details regarding the child placement history for the 50 case records reviewed. Children 
receiving case management services have had a multitude of placements, with 88% (44) of the children 
having at least one in-patient psychiatric admission. Some of the children have been in as many as 60 
foster placements over a 5 year period /see Table 5T 

Child Case Manager: Contact Hours 

Case managers spend the bulk of their time in contact with families and children, followed by contact 
with community services (e.g., school, out-patient community mental health) and residential settings 
tsee Table 6T Case managers spend a lot of their time on the phone. 

Analysis of the relationship between the amount of time case managers spend in contact with parents, 
children and other service providers, and the characteristics of children and families receiving children 
case management services revealed the following results. First, child case managers have the largest 
amount of total contact hours per family when the child has a history of placement in therapeutic foster 
care, and the least amount of total contact hours when the child has a history of a correction's placement. 
Second, Pearson R correlation coefficients were calculated for the frequency of the total child case 
manager contact hours by the child's score on the functional assessment inventory, the number of 
previous out-of-home placements, the total number of months in out-of-home placement, and the 
number of months from initial placement to most recent out-of-home placement. A moderate positive 
correlation was found between the total case manager contact hours and the child's score on the 
functional assessment inventory (R=.31, p=.029). All other child variables were found to have a 
minimum non-significant positive correlation. 
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Gender Differences 

Analysis of gender differences between boys and girls with SED were calculated for child, family, and 
case manager variables Csee Table 7). Mean scores on three variables were found to significantly differ 
between boys and girls. First, as noted in case records, boys, as compared to girls, had a higher number 
of family problems mentioned. Case managers spent more total contact time and total phone contact 
with families of boys, as compared to families of girls. 

Return to fop 



Summary 

Providing community-based child case management services is a labor intensive program with 
tremendous variability in the amount of time case managers must devote to helping families. While case 
managers spend an average of 2.5 hours per week in contact with a family (by phone, in person, and in 
community meetings), in some situations, case managers can spend as much as 20 hours per week 
helping just ONE family of a child with severe emotional and behavioral difficulties. 

The children and adolescents served by intensive child case management services are experiencing 
severe behavioral and emotional difficulties, and have experienced multiple placements and substantial 
time in placements. For example, children with a history of psychiatric hospitalizations had experienced, 
on average, 3 different placements with some children experiencing as many as 9 different placements 
over a 3 year period. The situation for children with a history of foster care placement is of even more 
concern. These children have experienced, on average 7 different placements over a 2 year period, with 
the outer range exemplified by one child who experienced 56 different placements over a 5 year period. 
Not surprisingly, as the total number of functional impairments exhibited by children increases, so does 
the total number of contact hours by the case manager. In other words, the more severe the child's 
behavior, the more time the case manager will spend with the child and family. 

Case managers are spending the bulk of their time with families, children and community-based service 
providers: the people for whom this program was expected to assist. Families, children and school 
ranked highest for use of case managers' time. Case managers work with families of boys more often 
than families of girls, and likewise spend more time with boys than with girls. More family problems are 
noted in the records of boys than of girls, with case managers spending 50% more time in phone contact 
with boys' families. Case managers spend substantial time providing services to consent decree class 
members. Class members, who are not in the custody of DHS, receive the largest amount of in-person, 
direct contact with case managers. Case managers also spend the largest amount of time in meetings 
related to class members. For class members who are in the custody of DHS, the situation is different: 
case managers spend the least amount of their time with this group of adolescents. 

Case managers work hard to return children, who have been psychiatrically hospitalized, to their 
communities. Children who are hospitalized while receiving case management services receive the 
highest number of total contact hours with the case manager: on average over 100 hours of service over 
a six month period. One of the major aims of the child case management program has been to assist 
families in creating an Individualized Service Plan (ISP) for their child. We are doing a fair job of 
meeting families' needs: a comparison of ISP services requested as compared to services received 
indicates that families receive the requested services 72% of the time, leaving families with unmet needs 
28% of the time, particularly in the area of respite care. 
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Outcome-Oriented Case Planning in Child and Family Services 



Child welfare agencies across the United States, Canada, Great Britain, Wales and elsewhere are 
working to better define the service delivery outcomes they are striving toward (e.g., American Humane 
Association, 1993; Rapp & Poertner, 1987; Parker, Ward, Jackson, Wedge & Aldgate, 1991). 
Outcome-oriented case planning is not new to child and family services or mental health, but recently, 
agencies have begun to develop ways to implement this approach more thoroughly in practice. 

To be most successful, the emphasis upon achieving specific outcomes must be present and supported 
throughout the provider organization, including line staff, supervisors, and managers. The process begins 
by determining how staff members assess children and families, set case goals, and develop service 
plans to meet those goals. Casework recording forms are critical to supporting a particular approach to 
service delivery planning and implementation (McDevitt, 1994). 

The Importance of Goal-Setting 

The human services literature has emphasized the importance of formulating clear and behaviorally 
specific treatment goals for many years. Research findings attest to the value of the social worker and 
client establishing agreed upon goals that will frame and guide their change-oriented relationship 
towards the desired outcomes. 

Goals specify what clients wish to accomplish (Cormier & Cormier, 1991). Inherent in goals are desired 
outcomes or changes in life situations that correspond to particular problems that have been identified 
through a focused assessment process. In long-term family foster care, the client is a youth who is being 
cared for by someone other than immediate biological parents. Setting goals for short and longer range 
outcomes is often a teamed effort between the foster parent, youth and the agency social worker. 

The establishment of behaviorally clear goals serves the following valuable functions for the youth, 
foster parents and social worker: 

• Promotes agreement among the youth, foster parents and social worker regarding the immediate 
and long range outcomes to be achieved, thus increasing the motivation of youth, foster parents 
and agency staff; 

• Specifies desired behavior changes more clearly, thus providing direction and continuity to 
helping efforts; 

• Facilitates the development and selection of appropriate treatment and intervention strategies; 

• Assists the youth, foster parents and social worker to monitor their progress towards the desired 
long range outcomes, so that each will know when and where achievements have been made, and 
where effort needs to be focused; 
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• Indicates the difference between what and how much a particular youth is able to accomplish, 
developmentally, in the near future and the distant future; 

• Permits agency staff members to determine if they have the skills, competencies, interest and time 
for working with the youth or whether there needs to be a referral of the youth to someone else for 
best service; and 

• Serves as outcome criteria in evaluating the effectiveness of specific interventions and of the 
overall helping process. 



Employing goal-setting to achieve the above functions requires knowledge about types of goals and 
skills in goal negotiation on the part of the line social worker. These are some of the core competencies 
necessary for implementing an outcome-oriented approach to practice. 
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Defining Long-Range Outcomes 

Accountability is a special responsibility for any program that sees its mission to be the "raising" of a 
child to adulthood. In programs geared to the provision of short-term services, for example, a 
concentration on short-term service goals may be appropriate. In a program that takes on a long-term 
parental responsibility for a child's development and future success, such a concentration is not adequate. 
Most long-term family foster care and group care programs have a number of long-range outcomes that 
guide overall practice with the youth and families they servei^ 

The primary outcome is helping youth to lead successful, independent lives by developing physically, 
emotionally, cognitively, socially and in other ways. For example, because The Casey Family Program 
(Casey) delivers long-term family foster care, its long-range outcomes are concerned with safe 
emancipation of youth; these outcomes are collectively are referred to as the Program Vision for 
Children. One or more of these long-range outcomes will be used to develop a Vision for a Particular 
Child which is constructed individually for each child in the care of the program. 

In plarming for a youth, case outcomes are the expected end states or developmental milestones to be 
achieved. A vision for a child includes the quality, or characteristics, of eventual adulthood seen as 
appropriate for that youth. The term case goal is used to designate the types of outcomes that generally 
may take a year to two years to accomplish. 

Case indicators are the more immediate changes or improvements expected to occur as a result of some 
action or service using a 1-6 month time frame. Indicators can also be thought of as the smaller 
milestones on the path towards accomplishment of case goals. The term indicator here is used in the 
more specific marmer adopted by many other human service programs which have implemented the 
mission&shyp; goal&shyp; objectives/indicators paradigm. 

Components of Case Planning 

There are at least three major components that pertain to professional case plarming in long-term foster 
care agencies. Some of these components are assumed and not explicitly described in program training 
manuals or operating guidelines: 

I. Assessments 

A. Child and family intake assessment; and 

B. Periodic case review and assessment 

II. Outcomes 

A. Program vision for children and families 

B. Vision for a particular child and family; 

C. Case goals for a particular child and family that are outcome-oriented; and 

D. Indicators that mark the progress made towards case goals; 
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III. Methods 

A. Development of the service plan (including case resources and case methods); and 

B. Services provision 
Return to Top 



Conclusion 

This summary describes the role of case goals and indicators within the larger case planning framework 
in family foster care, and these goals' relationship to assessment and services provision. As we move 
toward greater reliance on managed care structures for providing health, behavioral health, and even 
general child welfare services, we must continue to stress outcomes that are practice driven. If we don't, 
we will face systems that are driven by costs alone, without a basis for ensuring that we will take quality 
and program outcomes into account. 
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1 The Casey Family Program is a privately endowed operating foundation, established in 1966 by Jim 
Casey, a founder of United Parcel Service. The Program provides planned long-term family foster care 
to children for whom adoption and family reunification are not probable alternatives. After child 
admission to the Program, family reunification or adoption permanency plarming services are provided 
in response to a child and family's needs. Headquartered in Seattle, The Casey Family Program currently 
serves over 1,300 children in 23 communities in thirteen states. 
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System Level Outcomes 

Having examined the major outcomes of intensive case management (ICM) for children and their 
families in New York State, we investigated the major outcomes at the systems level. We believe there 
are four major positive outcomes at this level: (1) a decrease in state-operated hospital bed use, (2) cost 
savings, (3) the development of additional research projects, and (4) the refinement of the ICM program 



Hospital Utilization 

Being dependent on a one-group evaluation design, we used several statistical techniques to control for 
non-program variables that could explain the outcomes of ICM. One of these techniques is regression 
discontinuity analysis (Trochim, 1984). This technique was used to examine the state hospital utilization 
patterns of 917 children enrolled in ICM who had experienced state inpatient 
stays. We examined their hospital use for two years before their enrollment in ICM and two years 
following their enrollment. In Figure 1. the lines plot the average number of days used by children at risk 
of hospitalization during each time period. The analysis shows a decrease from more than 1 0 days (at 2 
to 4 months before admission) to about 2 days per month in the post-enrollment period. This represents a 
significant change (-8.2 days; p<.001). 



Regression discontinuity analysis, although one of the best types of analysis for single group designs, is 
open to criticism. Extreme groups such as the children enrolled in ICM may experience regression 
toward the mean. Therefore, we developed a post-hoc comparison group of children who had been 
admitted to state inpatient settings, but who were not enrolled in ICM, because this program was not 
always available to all eligible children. Comparison children were selected who matched an ICM client 
on age at first admission to the psychiatric center, gender, race/ethnicity, region of residence, diagnosis 
and total number of inpatient admissions, and whose total number of inpatient days was closest to the 
ICM client's total number of days in the two year period prior to that client's enrollment in ICM. 
Regression discontinuity analysis was conducted on the resulting sample of 784 children (392 x 2 
groups). The analysis examined 90 day intervals over a three year period of time (See Figure 2 ). 



The analysis showed increasing inpatient use for both groups during the pre-intervention period, but this 
was particularly true for the ICM group (7 days/quarter for ICM and 5 days/quarter for controls). This 
mean utilization decreases over time for both groups to less than 1 day/quarter for ICM children and 3.5 
days for controls. There is a significant difference in the slopes of the lines for the ICM group, 
comparing their pre- and post-enrollment periods. 



Cost Savings 

Regarding cost savings, the matched group analysis was used to estimate cost savings which could be 
realized from a decrease in hospital use by children enrolled in ICM. If the patterns of decreased 
utilization are annualized, children in ICM show a decline of 83% in inpatient utilization, from 
approximately 28 days in the year before enrollment to 5 days in the period between 2 to 3 years 
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post-enrollment. This compares to a decline of 34% for children in the comparison group. We converted 
this to bed days (22,109 bed days or 60.57 beds for the ICM group compared to 15,994 bed days or 43.8 
beds for controls during the year prior to ICM enrollment; see Figure 3T 

For the year period beginning two years after enrollment the data for ICM was 3,759 bed days 
or 10.3 beds, and for the controls, 10,556 bed days or 28.9 beds. The difference in savings that is 
accounted for by ICM is 12, 912 bed days annually or 35.4 beds. In terms of cost savings, using 
the children's state inpatient Medicaid daily reimbursable rate of $809.89/day, a decline of 35.4 beds 
used in a year can save $10,464,588. 

Research Projects 

The third systems outcome was the development of federally ftmded research demonstration grants that 
involve modification of the ICM program. ICM was targeted to children, not families, and their needs. 
We subsequently developed a Family-Centered Intensive Case Management model (FCICM) to be more 
sensitive to the context in which the child is located. For further information about this model, see 
Evans, Armstrong, Dollard, Kuppinger, Huz & Wood (1994). A short-term 
case management model (CCM) for children in psychiatric crisis also was developed, and we are 
conducting a study to compare the child and family outcomes associated with this model as compared to 
two family preservation models (see Evans, Boothiroyd, Armstrong, Kuppinger, & Bellair, 1995 [this 
volume]). 

Program Refinements 

In regard to the fourth systems outcome, program refinements, in New York State we are moving to 
develop and disseminate a best practices model of case management. We call this model Project SEED. 
This Project is an effort to combine the strengths of various community-based models 
of care including ICM and FCICM and to study its dissemination to counties with different 
characteristics, service systems, and populations. 

In summary, we believe that there have been significant, positive outcomes for the system of care in 
New York State that are associated with ICM. We expect to continue our investigations of systems level 
outcomes as we complete evaluative research associated with FCICM, CCM and Project SEED. 
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Figure 1 
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Figure 2 

Mean hpatient Days - ICM vs. Non-ICM Comparison Q-oup 
Pre-post ICM Enrollment Comparison - 90 Day Intervals 
(W= 784) 
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Introduction 

Intensive Case Management (ICM) for children represents the cornerstone of the community-based 
system of care for children with serious emotional disturbance in New York State. Implemented in 1988 
in various locations across the state, ICM combined high levels of service intensity with elements of 
individualized care to provide what was then considered to be a state of the art intensive, in-home, case 
management service for children who were at greatest risk of being removed from their homes. 

More specifically, ICM provides 24 hour/7 day a week response capability. The program established a 
small caseload, 10 children per case manager, to encourage more intensive service provision. Intensive 
case managers conduct comprehensive assessment of child and family service needs and link to needed 
services. Flexible service dollars are available to meet either individual child and family needs or 
identified needs of the collective caseload that are not otherwise available in the local system of care. 

The ICM model is an activist model which stresses the need for service delivery in non-office based 
settings, often in the homes of children and families. The model also promotes an advocacy role for case 
managers for the overall system and for children and families served. 

Implementation of ICM has occurred in stages, beginning in 1988 with an initial group of 64 intensive 
case managers with a capacity to serve 640 children, who were spread across 34 counties in all five 
geographic regions of New York State. Since then the program has grown to 185 intensive case 
managers operating in all but a handful of counties. 
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Methods 

To assess the nature of, and outcomes associated with ICM, the Bureau of Evaluation and Services 
Research at the New York State Office of Mental Health (NYSOMH) used three complimentary 
evaluation and data collection strategies. 

The first and most comprehensive is the collection of a minimum data set of descriptive data items on all 
children who are enrolled in the program. These data are completed by intensive case managers at intake 
and discharge. They enable the agency to understand the population served through collection of general 
demographic information about children, their treatment history prior to enrollment, and measures of 
their education, general functioning and behavioral status when they are enrolled. Data collected at 
discharge allow for the availability of proximal outcome assessments such as lengths of stay, discharge 
disposition and shifts in living situation between enrollment and discharge. 
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To assess outcome in greater detail, the second strategy initiated a longitudinal study of a 30% sample of 
children enrolled in ICM in 1990 was randomly drawn from the minimum data set database. Data from 
this sample of 199 children were collected to determine the child's status at enrollment, after 6 months, 
and at discharge or three years, which ever came first. This study assessed the impact of the ICM 
program on addressing areas of unmet service needs, child functioning and provider-perceived family 
strengths and family Actioning. Data presented from this aspect of our evaluation will focus on 
comparisons between baseline and the final data collection. 

The third evaluation strategy employs a matching strategy between the minimum data set and other 
secondary data bases available from the NYSOMH to assess patterns of inpatient service utilization by 
children receiving ICM services and matched comparison groups. 
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Results 



Table I presents data derived from the minimum data set on all children enrolled in ICM. To date, data 
have been collected on 2688 children served since early 1989. On average, ICM children are 12 years 
old. They are typically White and non-Hispanic, mostly male (67%), and live in a single parent 
household in the custody of a natural parent (75%). The typical child enrolled in ICM is educated in a 
special education setting (57%) and has a disruptive behavior disorder (45%). On average, ICM children 
have been out-of-home (either hospitalized or placed in a residential setting) two times, they have 
functional impairments in an average of 2.5 areas out of 5 assessed, and display an average of 5.7 
problem behaviors (out of a list of 25 areas assessed). 

Data on 1267 children who have been discharged shows that, on average, their length of stay is about 
13.5 months with a wide standard deviation. Comparative data on the living situation of children at 
enrollment and discharge tell us that 81% of the 710 children who lived in those settings at enrollment 
are there at discharge as well. 

Data from the longitudinal study which followed a sample of children through their ICM experience 
provide further insight into outcomes of ICM for children served. At the final data collection point, the 
study lost 37 cases for an attrition rate of 19% and a final sample size of 162. A logistic regression 
analysis predicting membership in the sample from the general population yielded no significant 
differences on any characteristic for which data were available, suggesting that biases introduced into the 
sample by attrition did not invalidate our findings. 



In addition to the minimum data set, data came from two other sources: the Baseline Data Collection 
Form, completed by intensive case managers, which supplemented the data routinely collected at intake; 
and a Follow-up form, which was completed by providers at either discharge or three years following 
intake. These data collection points included administration of the Child and Adolescent Functional 
Assessment Scales (CAFAS; Hodges, 1990) as a measure of child and family functioning on six 
dimensions, and selected scales from among the extensive set of Child Well-Being Scales (Magura & 
Moses, 1986) as measures of family functioning. A measure of child behavior status, the Child Behavior 
Checklist (CBCL; Achenbach, 1991) was also administered to parents as part of the study. 

The evaluation assessed changes in the level of unmet needs of ICM children in five life domains. 
Between baseline and discharge, unmet recreation (p<.01), medical (p<.05) and education (p<.05) needs 
declined significantly. Unmet mental health needs declined as well but not significantly. Unmet needs in 
the area of social supports remained unchanged. 



The study elicited perceptions of the child's symptoms and behaviors at baseline and again at discharge. 
A factor analysis of 25 symptoms/behaviors that were assessed resulted in the 7 factor groupings. In the 
areas of aggressive behavior (physically aggressive, verbally aggressive, destruction of property, temper 
tantrums, dangerous to others), suicidal behavior (dangerous to self, suicidal ideation, suicide attempt), 
anxiety (anxiety, depression, phobias, sleep disorders), 

fire setting/cruelty to animals, and psychotic symptomatology, declines were significant (p <. 01). Areas 
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of functional impairment as measured by the CAFAS show positive change in all areas with behavior 
towards self and others significantly improved (p < .05). Data from the parent completed CBCL show 
movement in a positive direction. On the total problem score the sample mean moved out of the clinical 
range between intake and discharge. 

Supplemental data collected on the sample provided some greater insight into the nature of the families 
of which ICM children are part. Families are characterized by high incidence of unstable relations 
between adult members, mental illness, poverty, adults who were themselves abused as children, chronic 
unemployment, alcohol/substance abuse, and domestic violence (see Table 2 ). 

Providers were also asked to assess family strengths which they felt could be built on to benefit the 
children. Measures of these factors showed great stability between the two data collection points with 
only significant (p < .05) change in greater recognition of families' ability to make their child feel loved 
and lesser ability to provide structure. 

Data on impact of ICM on families of children served showed no real change. The two CAFAS family 
resource scale s&shyp;Basic Family Needs and Family Social Supports&shyp;show no change between 
baseline and discharge, although mean scores on both (especially basic needs) showed low levels of 
problems in these areas. The Child Well-Being Scale scores used to measure shifts in family functioning 
also show little change between baseline and discharge. Only family's ability to recognize their child's 
problems improves significantly (p < .05). 

To examine length of stay (LOS) in ICM an ordinary least squares regression anal 5 dic model was used to 
assess our ability to account for variation. The strongest model had an R square of about .2. Variables 
that significantly contributed to the model are presence of abuse in family history (p < .05), the number 
of areas of functional impairment reported on a child (p < .05), level of family cooperation with service 
provision (p < .05) and, the strongest contributor, geographic region. In New York City after 1 .5 years, 
about 90% of children are still enrolled in ICM. In other regions for a comparable amount of time, 50% 
or fewer remain enrolled. In Figure I, a survival analysis displays this phenomena. 

To examine our ability to predict discharge disposition, we used a logistic regression approach including 
a dichotomized dependent variable&shyp;successful/unsuccessfiil termination. The sample for this 
analysis is reduced to 63 because about 1 10 of the 162 children were discharged at the final time point 
and there was some difficulty in accurately determining disposition in some cases. Among the 63, there 
is almost an exact split between successful and not. 

The model successfully classified 87% of cases. Variables significantly contributing to the predictive 
power of the variable were change in strength of the family relationship which reduces probability of 
successful discharge by 63% to only 29%. Table 3 presents the findings from this analysis. 
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Conclusions 

To summarize, findings from ICM evaluation activities suggest that, overall, ICM is having a positive 
impact on the children it serves. Data on child-specific outcomes show improvement after enrollment in 
ICM. Children identified as particularly needy are remaining in least restrictive, family settings. These 
positive outcomes are however tempered by findings suggesting that ICM has little impact on 
family-level outcomes. The importance of the strength of family relationship in predicting successful 
discharge and other data that show that case managers do spend much time working with families 
suggest that there may be a need to provide case managers with better tools to address family issues. The 
strength of regional variation in accounting for LOS variation suggest the need to investigate program 
differences and to examine the impact of ICM on the systems within which it operates. 
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Introduction and Purpose 

Through ongoing evaluation of statewide community-based programs (Huz, Evans & Dollard, in press; 
Huz, Evans, Rahn, & McNulty, 1993) and three active research demonstration projects. New York is 
actively engaged in children's mental health services research. It is logical that we now focus on 
understanding and documenting the processes through which these research programs can best inform 
practice and promote policy change. 

Project SEED (Services Evaluation and Dissemination) is a study of the process by which researchers, 
policy makers and practitioners disseminate recent research findings and innovative practice related to 
the provision of intensive case management services to children and youth with serious emotional 
disturbance. 

The specific goals of Project SEED are: (1) to summarize the outcomes and implementation experience 
of Project FIRST, the Children and Youth Intensive Case Management (CYICM) program and New 
York's efforts to operationalize Individualized Care principles; (2) to work in collaboration with service 
providers and parents to integrate these findings and design an enhanced model of CYICM; (3) to 
develop dissemination site selection criteria; (4) to implement the enhanced CYICM model in three 
sites, and (5) to describe the process of dissemination, the extent to which the components of the new 
model were implemented, and generalize about important factors in the dissemination process. 

Models for transferring knowledge derived from behavioral science research to practitioners have not 
been well developed (Lorsch, 1979; Astley & Zammuto, 1992). Consistent with this theme, policy 
makers and practitioners of children's mental health services consistently underscore the importance of 
research as a vehicle for modifying the system of care model and its component services (Rivera & 
Kutash, 1994; Stroul & Friedman, 1994; Weisz, Weiss, & Donenberg, 1992; Bums & Friedman, 1990). 
Recently there has been an increasing focus on services research and these studies are beginning to yield 
findings that can inform practice (Rivera & Kutash, 1994; Hoagwood & Hohmarm, 1993). Our 
understanding of and commitment to dissemination is critical, if widespread system change is to occur. 

The CYICM program model was developed in 1988 and it is appropriate six years later to revisit this 
model with input fi'om the CYICM practitioners, family members, the service provider community, 
researchers and policy makers. Children and Youth Intensive Case Managers carry caseloads of no more 
than 10 and are available 24 hours a day, 7 days a week&shyp;either directly or through a coverage 
system shared with other ICM's. ICM targets children and adolescents most in need through a local 
selection process known as rostering. Once targeted, a child is entitled to ICM for whatever length of 
time is needed. A critical component is the availability of flexible service dollars for meeting individual 
client needs. The goal of ICM is to enable the child to succeed at home, in school, and in the 
community. The CYICM model's effectiveness is well-documented (Huz, Evans, Rahn, & McNulty, 
1993; Banks & Evans, in press; Evans, Banks, Huz, & McNulty, 1994), and the practices of the most 
effective CYICM programs and providers will factor prominently in the design of an enhanced CYICM 
model. 



er|c 



269 



12/11/98 9:31 AM 



8th.251 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.25 1 .html 



In 1991 funding was obtained from NIMH for Project FIRST (Family-Focused Intensive Resources for 
Services and Treatment, NIMH R18MH5035/CMHS 5HD5SM50357). The project provided resources 
to compare child, family, and system outcomes for children referred and accepted for therapeutic foster 
care who are randomly assigned to either therapeutic foster care or Family-Centered Intensive Case 
Management (FCICM). FCICM, itself a modification of the original CYICM program model, has been 
successful in keeping children at home who would otherwise have been placed in a therapeutic foster 
home or other more restrictive placement (Evans, Armstrong, Huz, & Dollard, 1993). 

In the FCICM model, CYICM caseloads were reduced, a parent advocate joined the case manager as a 
member of the treatment team, and additional support services such as respite care, parent support 
groups, parent skills training, and flexible service funding (Dollard, Evans, Lubrecht, & Schaffer, 1994) 
were made available (Evans, Armstrong, Dollard, Huz, Kuppinger, & Wood, in press). An extensive 
battery of assessments and documentation of implementation issues from Project FIRST will inform the 
design of the enhanced CYICM model. The experience of FCICM also suggests that CYICM resources 
may be adapted to effectively service a population of children at particularly high risk of placement. 

Finally, the enhanced CYICM model will incorporate individualized care principles and practice. New 
York is committed through a number of initiatives to ensure that services focus on the individual child 
and family; provide services within a normalized environment; work in partnership with families; use a 
strengths-based, ecological orientation; demonstrate cultural competence; and be provided 
"unconditionally" 

In addition to the potential for improved service delivery at each site, this project will advance our 
understanding of a number of dissemination issues: (1) dissemination techniques (2) methodologies for 
measuring program model fidelity and adaptation, and (3) the relative importance of site, program, and 
population characteristics (Bauman, Stein & Ireys, 1991; Blakely, et ah, 1987; McGrew, et al., 1994). 
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Research Design and Methods 

This project takes place over a four year period. Year 1 involves preliminary work necessary to develop 
the enhanced CYICM program model, a dissemination plan, data collection and analytic strategies. 
During Year 2, the enhanced CYICM program model dissemination will begin in a pilot county. During 
Years 3 and 4, the enhanced CYICM model will be implemented in two additional counties for intensive 
study of the model dissemination and implementation process. Year 1 activities are described more fully 
below. 

Year 1; Development of the Enhanced CYICM Program Model 

The first year of this study involves the accumulation of information which will result in the 
development of an enhanced CYICM model and a plan for dissemination. Findings from the experience 
of implementing the novel program components of Project FIRST'S FCICM program will be 
documented and aggregated and contribute to an enhanced CYICM program model. Project staff will 
interview key FCICM participants to gather detailed information on FCICM services such as: involving 
a parent advocate in the delivery of service to children and families, incorporating respite resources, 
developing parent support groups, and providing parents with behavior management skills training. 

Data on the best practices of currently operating CYICM programs will also be collected during Year 1 
to assess innovative and successful program strategies. A survey instrument on program elements that 
have been found to be successful is now being piloted at two ICM sites. The instrument covers the 
following areas: admission criteria, family involvement, use of flexible service dollars, interagency 
collaboration, and discharge practices. 

A sample of 100 CYICM supervisors and case managers from 25 CYICM provider agencies will be 
selected to participate in the survey. The sample will be selected on the basis of findings from the 
evaluation of CYICM services. Criteria for selection will include geographic diversity, a well developed 
discharge policy, and a successful record of maintaining the most needy of the CYICM target population 
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in the least restrictive living situations. A sample of parents and youth will also be surveyed for feedback 
on their experiences with receiving CYICM services. 

Data from this survey and the Project FIRST experience will be analyzed and used to guide a series of 
focus groups involving intensive case managers, their supervisors, and parents of children enrolled in 
CYICM programs. The focus groups will be the final step in the development of a feasible enhanced 
CYICM program model and implementation plan. 

Data Collection Strategy 

Measures of child, family and provider characteristics, program fidelity, and service utilization will be 
administered over 1 8 months of initial program implementation in the two research sites. A Human 
Resource Questionnaire (HRQ) will be administered to assess characteristics of individuals directly 
delivering services to children and families. The HRQ has been used extensively in many evaluations of 
children's mental health services in NYS (Huz, Dollard, & Evans, 1990). 

The Child Description Form (CDF) and Program Termination Form (PTF) are extensively used to 
collect data on the demographic characteristics of children enrolled in intensive community-based and 
residential services in NYS. (Huz, Evans, & Dollard, 1994). These data will allow for comparison of 
characteristics of children served in enhanced ICM with children served in other sites. 

A Program Fidelity Index (PFI) adapted from an instrument and process designed by McGrew, Bond, 
Dietzen and Salyers (1994) will be developed to assess the degree of implementation of enhanced 
CYICM program components. The PFI will be administered to parents of children receiving enhanced 
CYICM, the service team (ICM's, parent advocates, respite staff), and program administrators. The 
degree of convergence from these multiple perspectives will reinforce the reliability of the research 
findings. 

The Program Contact Log (PCL), adapted from the Daily Contact Log developed by Brekke and 
Wolkon (1988), will elicit data on the degree and intensity of contact between service providers and 
children and families. 

The Service Involvement Survey (Research and Training Center on Family Support and Children's 
Mental Health, 1992) will be adapted and administered to parents and children receiving enhanced ICM 
services at the onset of the project and at 6, 12, and 18 months to elicit data from the parent perspective 
on enhanced ICM implementation. Items included in this form measure the intensity of service provision 
and key components of individualized care and cultural competence. 
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Significance of Study 

This study can feed 2 streams of more intensive services research: outcome research associated with 
enhanced CYICM and services research technology transfer. An assessment of the outcomes of 
enhanced CYICM including changes in child and family functioning, system effectiveness, and parent 
satisfaction could be the impetus to modify children's case management programs across the country. 
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Introduction 

Runaway and homeless adolescents present serious challenges to service providers. They are often 
victims of abuse, and as such they suffer from the developmental delays and emotional problems that 
would be expected. Others may be afflicted with behavioral or emotional problems that have interfered 
with their ability to stay connected with a non-abusive family. Dealing with emotional and behavioral 
problems while simultaneously trying to stabilize the living situations of these 
adolescents&shyp;perhaps reuniting them with their families or perhaps helping them to establish 
independent households&shyp;is a daunting task. On the face of it, it seems likely that professionals 
with mental health training and ample time to deal with each case should be able to do a better job than 
paraprofessionals burdened with substantially larger caseloads. 

We report here the results of a one-year follow up of runaway and homeless adolescents who were 
offered case management and mental health services at two levels of intensity. 
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Procedures 



Procedures and interventions are described more fully in Cauce, Morgan, Wagner, Moore, Sy, 
Wurzbacher, Weeden, Tomlin & Blanchard (1994). We provide only a brief description here. 

Subjects & Site 

YouthCare's Orion Center, in downtown Seattle, houses a variety of services for runaway and homeless 
adolescents: meals, recreational programs, food and clothing ba^s, a school program that is part of the 
Seattle school systems, a health clinic affiliated with the University of Washington, and mental health 
services supervised by the Seattle Mental Health Institute. The subjects of the study were homeless and 
runaway adolescents who approached the Orion Center for services. To be eligible for study 
participation, the youth had to be currently homeless, not in the physical custody of law enforcement 
agencies, at least 13 and not yet 21 years of age, not judged to be an immediate risk to the interviewers, 
and willing to agree to a series of interviews over the course of one year. 

Interventions 

Youth approaching the Orion Center for any services are assigned to a case manager who tries to 
connect the youth with appropriate services. "Regular" case managers have caseloads of 18 to 30 clients, 
and often have other duties. 



The experimental intervention consisted of a core of case managers with at least masters-level training in 
mental health issues, usually with an MSW degree. These case managers were provided with low case 
loads (maximum 12), access to psychiatric services for clients, and flexible funds for the clients' benefit 
(See Cauce et al., 1994, for more information). Youth agreeing to take part in the study were randomly 
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assigned to either "services as usual" or to the "intensive" condition. 

Measurements 

Study participants were interviewed at three-month intervals over the course of one year. The 
instruments administered included the Youth Self Report (YSR; Achenbach, 1991), the Reynolds 
Adolescent Depression Scale (RADS; Davis, 1990; Reynolds; 1987), the Problem Behavior Scale (PBS; 
Mason, Cauce, Gonzales, Hiraga, & Grove, 1994), the Personal Experience Screening Questionnaire for 
drug and alcohol use (PESQ; Winters & Henley, 1987), the Rosenberg Self-Esteem Scale (RSES; 
Rosenberg, 1965, 1979), and as a measure of life satisfaction, the Life Domains Scale (LDS; Baker & 
Intagliata, 1982). A variety of biographical information was collected; here we report only on 
victimization. 
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Results 

Three data points were examined on each measure for all participants with sufficient data. The baseline 
data points were based on the initial interview. Mid-point data were either from the six-month interview 
(first choice) or the three-month interview (second choice). Final data were either from the 12-month 
interview (first choice) or the nine-month interview (second choice). Repeated measures ANOVA with 
trend analysis was used on these three data points. One hundred and fifty youth were included in this 
analysis, 78 in the intensive condition and 72 in the regular condition. 

The results are summarized in Table 1 . As indicated in the "Group Differences" column, two of the ten 
measures (drug and alcohol use and reported victimization in the last three months) differed significantly 
between the groups at the start. We are unable to explain these differences as anything other than 
random sampling variation. The initial random assignment to conditions was rigorously adhered to. The 
results suggest that there were no significant differences in the ways the two groups responded to 
treatment, as shown in the "Group by Time" column. In nine of the ten measures, there were significant 
changes over time for both groups. These are shown in the "Time" column. All significant changes were 
improvements; nothing changed significantly for the worse. 

Figure 1 shows a graph of the changes in the Youth Self Report Externalizing scale. Both groups showed 
improvement with time; the improvement is statistically significant, and the lines are close to parallel. 
The very slight differences between the groups that appear upon inspection are not statistically 
significant when tested. This is the typical pattern for the results in Table 1. 
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Discussion 

Based on these results, we concluded that case management is effective in helping with the emotional 
and behavioral problems of runaway and homeless adolescents. The results do not show that specialized 
mental health training or artificially low case loads make a detectable difference in the overall 
effectiveness of these case management services. It appears that experienced paraprofessionals do as 
well as MSWs in this environment. 

There may still be differences in the ways that individuals with different presenting problems or 
different background experiences respond to the different services. This possibility will be investigated 
in future analysis. 
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Introduction 

The purpose of this symposium was to describe preliminary data evaluating empirically-based 
alternatives to the hospitalization of youth presenting psychiatric crises. The pages that follow 
summarize three presentations which addressed this topic area. 

The first presentation addressed the reasons for generating these alternatives, followed by a description 
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of a recently funded NIMH ROl study developed to determine whether an intensive family- and 
home-based treatment utilizing multisystemic therapy (MST) is a more effective and less costly strategy 
than hospitalization for addressing the mental health emergencies of adolescents with severe emotional 
disturbances (SED). 

The second presentation reported on the early outcomes of a federally-funded project in the Bronx, NY 
in which children presenting for psychiatric emergency services were randomly assigned to one of three 
treatment conditions. 

The last presentation described the methodology developed for the Alternatives to Residential Treatment 
Study (ARTS) and reported the preliminary findings from a sample of subjects in the first wave of a 
project evaluating longitudinal outcomes. Finally, the policy implications of these projects were 
discussed. 
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There is a widely held concern that children's mental health services have overrelied on expensive and 
restrictive inpatient treatments (e.g.. Bums & Friedman, 1990; Henggeler & Santos, in press; Office of 
Technology Assessment, 1986, 1991; Saxe, Cross, & Silverman, 1988; Sondheimer, Schoenwald, & 
Rowland, in press; Stroul & Friedman, 1986). Such overreliance presents several significant difficulties 
that have important implications for clinical practice and health policy. 

First, a grossly disproportionate percentage of mental health dollars are devoted to a relatively small 
percentage of youth who may or may not require such intensive services (Dougherty, 1988; Burns, 

1991). 

Second, this disproportionate allocation of financial resources might be justified in part, if youth treated 
in inpatient facilities were significantly more disturbed than their counterparts who are treated in 
outpatient facilities. Yet, studies indicate that approximately 50% of the youth treated in inpatient 
facilities do not have a severe or acute mental illness and that a proportion of youth presenting danger to 
themselves or others do not require inpatient treatment, especially if a home-based crisis intervention is 
available (Weithom, 1988). 

Third, there is no evidence indicating that inpatient psychiatric treatment of adolescents is more effective 
than any less restrictive treatment including no treatment (Bums, Thompson, & Goldman, in press; 
Office of Technology Assessment, 1991; Stroul & Friedman, 1986; Tuma, 1989). 

Fourth and finally, psychiatric hospitalization can have substantial negative effects. Hospitalization (a) 
fails to empower families, subsequently fostering dependence (Kiesler & Sibulkin, 1987; Sondheimer, 
Schoenwald, & Rowland, in press) (b) greatly restricts individual liberties and often pressures youth to 
conform to behavioral norms that have little counterpart in the outside world (Melton & Spaulding, in 
press); and (c) exposes youth to poor role models and labels them in ways that can impede future 
functioning (Henggeler, 1989). 

The central goal of this NIMH funded ROl study is to determine whether an intensive family- and 
home-based treatment (Crisis Family Preservation) is a more effective and less costly strategy than 
psychiatric hospitalization for addressing the mental health emergencies of adolescents with SED. 
Specifically, this study includes a 2 (treatment condition: Crisis Family Preservation vs. hospitalization) 
X 5 (time of assessment: within 24 hours of referral, post hospitalization, post family preservation, and 6- 
and 12 month follow-ups) design, with random assignment of yoked subjects to treatment conditions. 
Participants will include 252 12-17 year-old adolescents with SED referred to the projects by the Youth 
Crisis Stabilization Program of the Charleston/Dorchester Community Mental Health Center. Youths 
will be presenting severe mental health problems such as homicidal intent, suicidal intent, serious 
affective disorder or psychosis. A comprehensive multimethod, multisource evaluation will address the 
following questions: 

1 . "To what extent does Crisis Family Preservation services prevent hospitalization and reduce the 
symptoms precipitating the crisis?" 

2. "Over the long-term (12 months), to what extent does Crisis Faimily Preservation maintain 
reductions in identified symptoms; ameliorate the family, peer and community problems 
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associated with such symptoms; and prevent subsequent institutional placement?" 

3. "What factors moderate or mediate treatment?" 

4. "What are the comparative financial costs of the treatment conditions?" 
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Crisis Intervention 

Each year, approximately 55,744 children in New York State present at hospital emergency rooms 
experiencing a psychiatric crisis. Very little is known about the services they receive and the 
effectiveness of these services. The same can be said for children with acute psychiatric problems 
presenting nationwide. Although a number of intervention programs have been developed for these 
children and their families, little systematic data exist to evaluate the outcomes associated with these 
programs. 

In 1987, the New York State Office of Mental Health established Home-Based Crisis Intervention 
(HBCI) programs in five sites, including the Bronx. HBCI, an intensive in-home family preservation 
program, is New York's version of the Homebuilder's model developed in Tacoma, Washington 
(Kinney, Madsen, Flemming & Haapala, 1979). This program was designed to prevent out-of-home 
placement, including hospitalization of children in psychiatric crisis. The federally funded research 
demonstration study p^ational Institute of Mental Health (NIMH)/Center for Mental Health Services 
(CMHS)] discussed here sought to answer questions regarding the effectiveness of this program in 
comparison with a culturally competent, enhanced program (HBCI+) and a crisis case management 
program (CCSI). Specifically, the questions we sought to answer were: What program works best for 
whom? Can these services be used instead of hospitalization? What factors are associated with service 
choices for children in crisis and their families? 

The following table briefly outlines the salient characteristics of the three programs being studied in this 
research. All are intensive, in-home interventions which are typically four to six weeks in length. 

Based on the logic model for this research, we are collecting data in the areas of child characteristics, 
family characteristics, provider characteristics, program characteristics, child outcomes, family 
outcomes, system outcomes, and provider outcomes. 
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The services demonstration involved the establishment of HBCI+ through enhancing the basic HBCI 
model with respite care, flexible service money, a parent advocate, and staff training in cultural 
competence, as well as working with families who may be exposed to violence within the family or 
larger environment. The Hispanic Research Center provided the training in cultural competence, 
translated instruments into Spanish, pre-tested the instruments and was responsible for data collection in 
children's homes. Training regarding intrafamily violence was provided by Earl Braxton and colleagues 
and training on strategies for raising children in socially toxic environments was provided by James 
Garberino. Crisis Case Management (CCM) was developed from New York's Intensive Case 
Management program by providing staff training in dealing with children in crisis, providing respite 
care, and by decreasing the caseload of case managers. 
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Method 

The research uses a three group positive-controlled randomized study design in which eligible, 
consenting children/families presenting to two of the three emergency intake sites in the Bronx are 
randomly assigned to one of the three treatment conditions (HBCI, HBCI+, CCM). Eligible children are 
those 5 to 17 years of age, living in the Bronx at home with a natural, foster, or adoptive family, who are 
experiencing a psychiatric crisis requiring immediate intensive intervention. In the absence of these 
services the child would likely be hospitalized or placed in a restrictive setting. Finally, it must be safe 
for the child to return home with intensive services and the family must be willing to receive in-home 
services. Data on the child and family characteristics and functioning are collected on intake, at 
termination from the intensive program (4-6 weeks), and at 6 months post- termination. Data are also 
collected on all children presenting to the psychiatric emergency settings to allow comparison of those 
hospitalized, referred to in-home services, and referred to other less intensive services in the community. 
Return to Ton 



Results 

The data presented are for the first 125 children/families enrolled in the intensive in-home interventions 
and the first 671 children presenting to the emergency settings. The children average 12.4 years (range 
5-18) with equal proportions of males and females. The majority were of Hispanic origin (64%), with 
30% Black/ African-American and 6% other. The most common diagnoses were disruptive behavior 
disorders (30%), with 18% presenting with adjustment disorders, 14% with psychotic disorders, and 
17% with mood disorders. Clinicians noted that 77% were experiencing functional impairment due to 
the disorder and 86% demonstrated behavior dangerous to self or others. Of these children, 51% were 
enrolled in special education programs, 40% had received prior mental health treatment and 9% had 
been previously hospitalized for a psychiatric disorder. 

Most of the children (76%) were being raised in single parent households of three or more children 
(55%). Few mothers had completed high school (25%) and few were working (25%). Most received 
assistance from Medicaid (65%) and/or Aid to Families with Dependent Children (36%). More than half 
the families had an income less than $10,000 per year. 

Figure 1 shows the overall dangerousness of the child's condition as rated by clinicians using a risk 
assessment instrument developed by Gutterman and Levine (1992). Significant differences in the 
assessed dangerousness existed among children referred to each of the three interventions. 

Children admitted to the hospital were most at risk clinically, had the lowest capacity for cooperating 
with a treatment plan and had the lowest parental support, while those in the in-home condition were 
intermediate, and those referred to other community programs were rated least at risk on all three 
subscales. 
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Looking at data for the first 86 children to complete the in-home intervention indicates that they 
experienced a significant gain in self-esteem from intake (Mean = 47.4) to discharge (52.5) as measured 
by the Piers-Harris Children's Self-Concept Scale (Piers, 1984). Beginning with high t-scores on Total 
Problems, Internalizing and Externalizing subscales as measured by the Child Behavior Checklist 
(Achenbach, 1991), these children did not experience a decrease in their scores on discharge from the 
intensive programs. 

Regarding family data, scores on cohesion and adaptability, as measured by the Family Adaptability and 
Cohesion Scales (Olsen, Portner & Lavee, 1985) increased significantly from admission to discharge 
(see Figure 2 ). 

The family's social supports, measured by the Inventory of Socially Supported Behaviors (Barerra, 
Sandler & Ramsey, 1981) indicated no change from admission to discharge. What is most striking is that 
the social support scores were so low for families enrolled in all three programs, indicating that families 
were not only financially, but socially impoverished. 

Finally, regarding parent satisfaction with the treatment program, parents in all three conditions reported 
a high level of satisfaction with services. It is interesting to note, however, that there was an ordered 
effect regarding satisfaction with skills gained. Parents in CCM were least satisfied, those in HBCI were 
intermediate and those in HBCI+, as we hypothesized, were most satisfied. 
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Discussion 

This project is still in an early stage. We will be enrolling children/families until we have 260 families 
completing the intervention. To date, however, we have learned several things. The first is that children 
in psychiatric crises can be cared for safely in their homes, given a sufficiently intensive intervention. 
Secondly, left to exercise their clinical judgment, clinicians differentiate among children referred to 
inpatient, in-home and community treatment settings. That is, they make decisions regarding the child's 
placement based on their assessment of the severity of the child's condition and the child and caregiver's 
ability to participate in a treatment plan. The data indicates that the in-home programs may not be 
ftmctioning to divert hospitalization, one of the original goals for these programs. A sizeable proportion 
of the children assessed (20%) are referred for hospitalization despite the availability of intensive 
in-home services. 
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Introduction 

As the public sector becomes more and more driven by the requirement to demonstrate accountability 
for its policies and programs, the children's mental health system will be required to support policy 
decisions with favorable outcomes. As Bums and Friedman (1990) have indicated, empirical data in the 
children's mental health field is sparse. Furthermore, it is a new field with emerging methodologies and 
measurement instruments. In this context of pressing need and evolving research strategy, the 
Alternatives to Residential Treatment Study or ARTS was implemented. 

While a randomized clinical trial would be the best approach to evaluate the efficacy of alternatives to 
residential treatment, we determined that available resources and currently available instruments could 
not adequately support such a study. Consequently, we attempted to design a systematic, descriptive 
study that would give important information to the field and contribute to the Imowledge base. The 
system of care framework developed by Stroul and Friedman (1986) had a major influence on the 
CASSP initiative and offered a focus for ARTS. 

By the early 90s the conventional wisdom about treating children with serious emotional disabilities 
began to shift in the location of treatment. Previously, it was assumed that intensive treatment could only 
be delivered in a hospital or institutional residential setting. This assumption was being challenged by an 
emerging collection of programs that attempted to keep the child in a community setting, preferably at 
home or in a home-like setting. The Kaleidoscope Program in the inner city Chicago area emerged as a 
leader through many training activities for the CASSP community and then through the adoption of its 
program principles by Alaska and Vermont for state- wide program implementation. Another important 
influence at the time was the Ventura County Children's Demonstration Project in California. This 
program served as an example of cross agency, collaboration to implement an integrated service delivery 
system with flexible funds and community based programs. The Therapeutic Foster Care Program 
offered by the Pressley Ridge Youth Development Extension (PRYDE) was another influential program 
in terms of its training and research activities. These programs offered a natural laboratory to examine 
the emerging models of family centered, child focused treatment models that were alternatives to 
traditional institutional placement. 

The design we planned aimed to answer some important questions in the field. These questions were: 
What are the characteristics of the children served in these programs? What are their social service 
histories? What services are delivered in these programs? What happens to children in these programs 
over time in terms of their symptoms, cognitive and emotional functioning, and living conditions? 
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Results and Discussion 

The ARTS protocol and procedure have been fully described in an earlier publication (Duchnowski, 
Johnson, Hall, Kutash, & Friedman, 1993). The protocol was administered at program entry, six months, 
and twelve months after admission. There were 144 children in the study. 

These programs serve youths who have multiple problems that are of a long duration and severe in 
intensity. With an average age of slightly over 14, there were 61% who had contact with the justice 
system for an average of eight times. They had an average of four placements in mental health 
residential treatment centers and only 36% entered the current program from their home. 

Involvement with the child welfare system was common. Sixty-nine percent of the families had worked 
with a child protection worker because of a charge of abuse or neglect. In addition, 55% of the children 
in the study had been placed out of their home because of a charge of abuse. 

The group as a whole had very poor academic performance, with 80% below grade level in reading and 
90% in math. Not surprisingly, 80% had a past history of placement in special education programs. 
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This is a group of youngsters that was well known to the major child serving agencies, i.e., mental 
health, child welfare, juvenile justice, and special education upon placement in the programs that we 
studied. The average age of problem onset was reported to be 6.8 years while the average age at the time 
of first service was 8.7 years. On the average, almost five more years would elapse before the children 
were placed in the current program, a time during which there were multiple contacts with multiple 
agencies. 

There was an unexpectedly low number (36%) of children who were living with their family at the time 
of program entry. Since all of these programs valued family preservation, we anticipated that we would 
observe extensive work with families. However, the high percent of children in state custody and in 
out-of-home placements indicates that many of the children were already estranged from their families 
before entering the programs being studied. This may have contributed to the finding that only 34% of 
the children were in the custody of a family member after 12 months in the program. Many of the 
children had not been involved with their family for a long period of time. 

Child Functioning Indicators. The results from measures of emotional and behavioral functioning 
(CBCL & YSR), and functional impairment (CAFAS), are very encouraging. Scores at the 12 month 
interval were significantly lower than at program entry, indicating a reduction in emotional problems 
and functional impairment. Results from the RBPC also indicated improvement in this domain but only 
the conduct disorder scale yielded a significant difference. The extreme level of emotional problems in 
these children is indicated by the finding that even though their scores on the CBCL are significantly 
lower after 12 months of treatment, the mean is still in the clinical range. The mean for the YSR, 
however, below the clinal range at post-test. 

Scores on measures of academic performance taken at the 12 month point are also encouraging. While 
the average achievement level may still be below expected grade level, the means for both reading and 
math were significantly higher after 12 months. Results from testing in these programs indicate that 
children with serious emotional disabilities can improve academic performance and reverse the 
downward spiral of poorer and poorer test scores. All of the other scales such as self esteem and the 
SSRS yielded results that indicated improvement after 12 months but the differences were not 
statistically significant. Likewise, the measures of family functioning and impact on family yielded 
non-significant differences though they showed improved status. The results from the latter two 
measures may have been influenced by the small number of families available to complete these scales. 

Services and Placement Outcomes. There were 47 different services that children in the study received. 
These ranged from out patient mental health to supervised living services. We reduced the 47 different 
services to 1 1 different categories and found that 80% of the children received between four and seven 
different categories of service. All children received case management. 

After 1 2 months of treatment, we evaluated the restrictiveness of living arrangement and school 
placement. In terms of where they lived, 40% were in less restrictive settings, 32% at the same level, and 
28% were living in a more restrictive setting. In school, 31% were in a less restrictive setting, 56% were 
at the same level, and 13% were taught in a more restrictive setting than at program entry. 

These initial results are viewed as promising and indicative of improvement in the functioning of 
children who have serious emotional disabilities when treated in a community based program. The 
results should encourage continued examination of community based services employing rigorous 
research designs. 

Return to Top 
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Introduction/Purpose 

In 1993, the Nebraska Department of Social Services contracted with the Center for the Advancement of 
Human Development (the Center) to design and implement community-based mental health/child 
welfare services in two service districts in Nebraska. The purpose of this contract was two fold: (1) 
return adolescents who were placed in out-of-state institutions; and (2) prevent adolescents from being 
placed in out-of-state institutions by serving referred youth. The Center was also required to identify 
psychosocial variables associated with various aberrant youth behavior and identify the level of 
community tolerance of those behaviors. 

In response, the Center developed two group living programs, specialized therapeutic foster care, 
home-based therapy, and family support services. The Center developed the capacity to receive 20 
adolescents into care within the first 40 days of operation and developed 10 therapeutic foster care 
families within 10 months. In April, 1995, the Center began the process of acquiring 23 family support 
workers to provide a total of 3,680 hours of family support services per month, and is implementing an 
in-school wraparound program. Additionally, the Center was awarded an expansion contract to include 
two additional districts in Nebraska. 

To develop the service system, the Center and the Nebraska Department of Social Services agreed to a 
purchase of service arrangement beginning on day one of the contract. System financing was 
accomplished through a 60 day pre-payment consideration with very aggressive utilization rates and 
outcomes established as contract stipulations for the two year contract period. 

Fiscal outcome data strongly demonstrated significant cost savings to the State of Nebraska. Clinical 
outcome data strongly supported the inclusion and integration of youth into community programs 
including public education, and the further use of less restrictive levels of care for this population. 
However, data also indicate that certain youth and adolescent behaviors present serious barriers to 
community inclusion efforts. Identifying the variables associated with various child behaviors has 
permitted an early identification of those children who show a very strong probability of experiencing 
extreme difficulties in school. Extreme behavioral difficulties in school were predictive of community 
exclusion in favor of a placement in a locked correctional facility. Prior to this project, program 
evaluators were limited to capturing this information after the child had been excluded. This project has 
provided a mechanism to predict which children will experience extreme levels of behavioral difficulties 
in order to develop intervention strategies before the child enters school. 

Return to Tod 



Method/P roced u res 



Service Outcomes as Contract Stipulations 

The Nebraska Department of Social Services and the Center agreed that much of the service system in 
Nebraska has been developed in the absence of empirically derived information. Programs have been 
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funded, children have been placed, and policies developed based largely on lobbying efforts, story 
telling, and clinical intuition. In an effort to change the 'standard operating procedure,' the Nebraska 
Department of Social Services and the Center agreed to include service outcome measures as contract 
stipulations in an effort to establish strategies to promote empirically derived information sources which 
could inform multiple users of service, policy makers, clinical professionals, and others interested in the 
issues of treating children with emotional disturbances. In that spirit, the Center and the Nebraska 
Department of Social Services agreed to the following: 

1 . The Center would build a service system which would be comprised of group living, therapeutic 
foster care, intensive family preservation services, and school supportive services in two rural 
districts to respond to the individual mental health needs of children who were placed out-of-state 
in long-term psychiatric care facilities, and to return those children to the home districts. All 
employees of the system would be hired from the local communities. 

2. As an incentive to return these children as soon as clinically possible, the Center received 60 days 
pre-payment on a two year contract as a function of program initiation. The Center would be 
required to maintain a utilization rate better than 96% after the first 40 days of program service. 

3. All youth and adolescents would be served by the local public education system, with the 
expressed desire that all children be fully integrated into regular education classes, with special 
education supportive services provided within the regular education setting if required. 

4. Empirically derived program as well as child/family indicators would be developed and fully 
implemented with the aim that those indicators and outcome variables would be used to identify 
precise difficulties in serving these and similar adolescents and families in rural settings. 

5. The Center would maintain a "no reject" policy regarding children admitted to the system of care. 

6. The Center would develop further program components at discharge based upon the identified 
needs of each child (wrap around model of service provision; independent living). 

Subjects 

The Center obtained a complete set of data on each adolescent served. The first program component 
'on-line' was the group-living facilities, which served all of the children returned from out-of-state 
institutions. All adolescents categorized as the prevent from going out-of-state' group received services 
from the group-living facilities as well. Child demographics are shown in Table 1 . 

It should be noted that the Group Living programs also provided 213 days of emergency and temporary 
(less than 15 days average length of stay) care for an additional 19 adolescents. 

The adolescents served were primarily older, with long histories of placements in very restrictive 
placements. Many of these youth did not possess age appropriate social skills, or problem-solving 
strategies. In the face of confrontation or frustration, they would frequently resort to temper tantrums or 
destructive out-bursts, followed by requesting 1:1 therapy sessions. Most of the children had not 
attended a public school for more than two years. 

Return to l op 



Measurement/Instrumentation 

In an attempt to identify problem-prone child behaviors associated with the two most influential 
contextual settings in which youth must participate, the Center employs the following instruments: 

1. Daily Adjustment Indicators (DAI; Burchard, 1990). The DAI is a 25 item (binary) behavior 
checklist designed to be completed by adult caretakers on a daily basis across all living 
environments. 
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2. Eyberg Child Behavior Inventory (ECBI; Eyberg, 1992). The ECBI is a 36 item behavior 
checklist that allows the adult to rate each child behavior from never to always and permits a 
second rating per behavior on a second adult dimension; Is this [behavior] a problem for you? 
Adult direct care providers complete this checklist monthly. This inventory yields a total problem 
score and a total intensity of behavior score for each child 

3. Sutter-Eyberg Student Behavior Inventory (SESBI; Sutter & Eyberg, 1992). The SESBI is a 36 
item behavior checklist that allows the teacher to rate each child behavior from never to always. 
This inventory also rates a second caretaker dimension: Is this [behavior] a problem for you? 
Teachers complete this checklist every other month. As with the ECBI, measures include a total 
problem score and a total intensity of behavior score for each child 

4. Community-Oriented Programs Environment Scale (COPES; Moos, 1987). This instrument, 
designed for community-based treatment programs (e.g., board and care homes, sheltered 
workshops, and halfway houses), includes 100 items, 10 subscales, normed on 74 programs. 

Youth rate their living environment using COPES every other month. 

5. Achenbach Youth Self-Report (CBCL; Achenbach, 1983). Youth complete this 1 12 item behavior 
problems checklist at intake and discharge. 

Return to l op 



Results and Implications 

By returning these targeted adolescents, the State of Nebraska received an annualized savings of 
$1,067,621.00. In other words, in the absence of this service, Nebraska would have spent an additional 
$1,067,621.00 during the fiscal year. These actual savings were used by the Nebraska Department of 
Social Services to 'sell' the service system to policy makers, and argue the legitimacy of alternative 
levels of care for very seriously involved adolescents. 

Preliminary data analysis included performing various correctional analyses, analysis of mean scores, 
and regression analysis. Very early in this work it was determined that separating the data by 'time in 
service' provided very useful information. 

The children and youth demonstrated sufficient variability from month to month on the Daily 
Adjustment Indicators and the COPES. A description of the DAI can be seen in Table 2. 

It is important to recognize that the first month data does not represent a full month 'in care.' It represents 
the month the youth entered the program. In other words, the data is representative of some portion of 
the total days of the month the child was in care. The second month actually represents the first full 
month of data /see Table3~). 

The COPES permits an analysis of the child's perception of his/her living environment. Generally, the 
lower the score the less the child is engaged in his/her living situation. 



The regression analysis was designed to answer the following two part question: Which criterion 
variable, (the youth's Sutter-Eyberg Student Behavior Checklist score or Eyberg Behavior Inventory 
Intensity /Problem score), and what set of predictor variables (DAI) and (COPES), would result in the 
best predictive model? 

Being able to answer the above complex question could equip program staff to quickly evaluate a 
youth's potential for problems in different environments based upon an earlier assessment. These 
instruments are completed by the youth, the program staff, and school personnel, thus representing 
multiple informants in different settings. Provided that a meaningful model could be built, such a 
statistical model could promote more comprehensive clinical and program decisions. 



For example, it was determined that in general, elevated Physical Aggression scores and elevated Verbal 
Abuse scores in the residential setting were predictive of serious problems in school (Sutter-Eyberg 
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Behavior Checklist -Intensity of Behavior Score). In other words, if children, demonstrated elevated 
scores on these DAI dimensions, they tended to have extreme difficulties in the education settings as 
well. It should be recognized that this pattern held true regardless of age or community in which the 
youth was served. 

The second analysis involved the COPES scores with the DAI dimensions against the Sutter-Eyberg 
scores. A slightly different but more complete picture emerged after this analysis. There was an 
interaction effect between the COPES scores and the two DAI dimensions. However, Physical 
Aggression scores at month two and three and the COPES score tended to be the best predictors of 
behavioral difficulties in the educational setting. Children scoring the lowest on the COPES tended to 
score the highest on the Physical Aggression dimension, and had the most difficulties (behaviorally) in 
school. 

Our fiill analysis indicated that the children scoring the lowest on the COPES tended to be the children 
that obtained the highest scores on Physical Aggression at months two and three; had extremely elevated 
scores on Verbal Aggression, had the highest Sutter-Eyberg Intensity Scores, and did not remain in 
school, or the community. In fact, the children who statistically fit this profile were eventually targeted 
first by the school; referred to the county attorney's office by school personnel, received a new petition 
(delinquent) by the county attorney, and placed in Nebraska's correctional facilities after a hearing by a 
county judge. 

It should be noted that each of these targeted children had past delinquent charges, and were not citizens 
of the communities were they received residential or educational services. 

Return to Top 



Discussion 

The current data indicate that these children did act very differently from the other children in care, 
regardless of the county of origin. However, the associated 'characteristics' are important considerations 
at this point in the analysis. What defines these children as a 'set' vs. other children whose scores are 
similar becomes critical when developing both clinical interventions and administrative positions. Were 
these children treated differently somehow? If so by whom? Did these children fail to 'fit' into the 
community's picture of youth who should be served by community services? 

Clearly, the educational settings played an important role in the identification and referral process to the 
county attorney's office. If providers are to successfully serve troubling children, it is important to be 
able to recognize those children who demonstrate an elevated risk of being removed from communities, 
and to introduce interventions aimed at changing that trajectory. Educational settings represent important 
spheres of community infiuence&shyp;as schools began to encounter difficulties beyond their ability to 
manage, they may engage the local county attorney's to invoke various legal mechanisms to relieve the 
pressure. These mechanisms can result in charges being filed on targeted youth and placement in 
correctional facilities, by-passing less restrictive interventions. 

The local school authority's use of the county attorney's office appears to a method of defining the 
parameters of acceptable adolescent behavior and asserting the community threshold of tolerance. 
Although it is too early to clearly articulate the behaviors associated with that threshold, it is obviously 
interactive, and subject to change through social intervention at the school level. With the accumulation 
of additional data a more precise picture will emerge. 
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Tablet 

Youth Demographics: Group Living 



Sex: 

MiOes 21 

Female s 14 

Ages: 

10-13 yre 3 

14rl6yis 22 

17-19 yre 10 

'FaxxL 

Cuicastui 34 

Me)dc 2 ui/Ameiicaii 1 

IxnmedistiQ fiioirpiaGeirien.t 
Outol-stite locked 

psychiaizic feciMes 9 

In-data locked psychialrr tacilities 6 

OihierLocked lesidentiaL 
tieaimentiaciliius 2 

In-data long-tem porate 
psychiatdcDciliius 4 

Emc me Ticy shelter/ 

acute JLO^itdisdion 10 

Foster care 2 

Fekdive 2 

Type of lawvioJatiatB vftrile inourcaie: 

Shopliftiiig 4 

Assault 3 

Criminal misichiet 4 

hjfajordiagniosticCTtegoqAes repiesentad: 

Conduct Disorders 9 

C^osdtional De 9 

M^orDepiession 9 

PostTzaumatic Stress 8 

Eipol 2 tfDisozder 5 

Attention Deticit Disorder 4 

SchiiDi^iEnk 3 

Antisocsd Personalia 2 

Borde z^ine Personalty 2 

Nok: uiddtBai^laa iuilidi: 



piop ily di rbrntflaM, SKul«JtiEg«>i.% 
si^bIk iSsfllioayB^ ztui^ IwAt ii£i]pKadc a.1 Zbag 
pe«apii» Mdtioa^ laolt aojua% 



Tablet, continued 



SiuQtioizBl seivioes provided by 
locd public schools: 



GKD 


2 


Maizistzeam regular education 


16 


Seli-conitained classzDom. 


2 


Maizistzeam with %>ecial Ed. support 

N«fe: QBeaepj«d«al 


14 


Disdiarge jlaaexnents 
(average length of stay 7 months): 




Indepe ndent living : 


5 


Fjelativtes 


S 


Foster care 


4 


Detention 


2 



N«^: B Ic ■«% d <kc CaMp rvo^pasgiAixE b 1 b» 

joteoaflcd Z13 d^|B«iK2zui^a^ad liB^eoApCk bIBa 13 
lc& A addilaAdl 10 xsa.it 
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Table 2 

Var iabi I ity of Ch i Ure n and Youth on Da ily 
Adjustment Indcator 



Ektieitie VerVal Alxue: or yaai!k 

svetk toaxati^pefsofi n o» 

f^sivear intbmdatiJitT»<oineyf 

month 1 month 2 month 3 



mean= 1.27 mean= 3.55 mean= 459 

min= 0 min=0 min= 0 

maK=5JD0 maK= 12.00 maK= 14.00 



Fh;raical Aggnessioor Did the diid or 
$bihe,^ite,arso^diajrerscii laom 

(in dudes at ct^esi.)f 



month 1 month 2 month 3 



means P5 means 2.00 means 1.95 

mins 0 mins 0 mins Q 

maxs 12.00 maxs 12.00 maxs 11.00 
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] 



Tables 

COPES Analysis of Child’s Perception of 
Living Bnvironment 


Meari = 


4377 




(noimed soreiage = SOD) 


ad Dev. = 


985 




(iwnn«dSW.Dev,= 10 D) 


Mixiixnmn= 


27D0 


Ma^diYium = 


63D0 
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Overview 

Each year, numerous children, adolescents, and their caregivers seek help in psychiatric emergency 
settings. Although the mental health field lacks standardized approaches to treat these children and to 
support their families (Rosenn, 1984), short-term, intensive, in-home interventions modeled on the 
Homebuilders program (Fraser, Pecora & Happala, 1991) are being widely disseminated. 

In 1987, the New York State Office of Mental Health (NYSOMH) established Home-Based Crisis 
Intervention (HBCI) as an intensive in-home service option for families. HBCI is modeled on the 
Homebuilders program, which provides short-term services to families with a child experiencing a 
psychiatric crisis. In 1993, the NYSOMH received a three-year research and demonstration award from 
the Substance Abuse and Mental Health Services Administration (SM-50357) to study the delivery of 
in-home crisis services. In this project, two new intensive in-home psychiatric emergency programs for 
children and their families&shyp;an enhanced version of Home-Based Crisis Intervention (HBCI+) and 
Crisis Case Management (CCM)&shyp;were implemented and are being compared to the existing HBCI 
program. In-home and out-of-home respite services are additional supports available to families assigned 
to the HBCI+ and CCM conditions. This summary describes the respite services available to parents in 
the HBCI+ and CCM conditions, examines the use of respite in these programs during the first year of 
the project, identifies the barriers to the use of respite raised during a focus group conducted to increase 
respite use and the steps implemented to minimize identified barriers, and compares the profiles of 
children and families who use and do not use respite. Only families who completed the foil intervention 
are included in this analysis. 

Return to Top 

What are the respite services available to families as part of this study? 

In-Home Respite: For families enrolled in the HBCI+ and CCM programs, in-home respite has been 
made available on both an emergency and planned basis. In-home respite is provided by workers 
specifically trained to care for children with significant mental health needs. Services to a family are 
typically provided by the same respite care provider who is selected to match the needs of the child and 
the family. Given the cultural diversity in the Bronx, English and Spanish speaking respite care 
providers are available. In-home respite services are also not required to specifically take place within a 
family's home. Respite care providers can engage in recreational activities with a child such as going to 
the zoo, seeing a movie, or taking a trip to the mall. A separate pool of money is available to cover the 
costs associated with such activities. Additionally, in-home respite services are not restricted to the 
identified child (i.e., the child receiving either HBCI+ or CCM), but also are available for his/her 
siblings. 

Out-of-Home Respite: Out-of-home respite care is also available to families enrolled in the HBCI+ and 
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CCM programs. In out-of-home respite, a child is taken into the home of a trained respite care family. 
Out-of-home respite providers were hired and trained by a private agency licensed by NYS to provide 
such service. Given the short-term nature of the intervention in this project, out-of-home respite stays are 
limited to three days per occasion, but there is no limit on the number of instances in which it may be 
used. As is the case with in-home respite, out-of-home respite is available on both an emergency and 
planned basis. Respite is also provided by the same respite care family, to the extent possible, for 
families using out-of-home respite on multiple occasions. As possible, respite homes are selected to 
match the needs of the child and the family and both English and Spanish speaking respite providers are 
available. Out-of-home respite can also involve extended days that do not require an overnight stay. 
Return to Top 

How has the use of respite during the first year of the study corresponded with prestudy 
projections? 

In-Home Respite: It was projected that 3,480 hours of in-home respite would be used during the first 
year of this project (58 families x 6 weeks x 10 hours per week = 3,480 hours). As shovm in Figure 1, 
examination of the 59 families enrolled in the two study conditions (CCM and HBCI+) with access to 
in-home respite indicates that 16 families (27%) used this service. These 16 families used 847 hours or 
24% of the available in-home respite hours (see Figure 2). The number of hours of in-home respite 
services that families received ranged from 2 to 106 hours, averaging 41 hours. 

Out-of-Home Respite: It was projected that 400 days of out-of-home respite would be used during the 
first year of this project. The same 59 families enrolled in CCM and HBCI+ during the first year also 
had access to out-of-home respite. Three families (5%) used this service (see Figure 1). These 3 families 
used 1 1 days or 3% of the available out-of-home respite days. As shovm in Figure 2. the number of days 
of out-of-home respite services that families received ranged from 2 to 5 days, averaging 2 days. 

During the first year of the study the use of both in-home and out-of-home respite fell below anticipated 
levels. Over two thirds of the families with access to in-home and out-of-home respite did not use either 
service. The majority of the families using respite used in-home service. Only three families used 
out-of-home respite. 

Why has the use of respite during the first year of the project been below anticipated levels? 

As part of the project data collection schedule, parents are interviewed at discharge. During the 
interview, parents whose child was enrolled in one of the two programs with access to respite are asked 
about their knowledge of these services. An examination of their responses indicates that nearly 62% of 
the parents remembered being informed about the availability of in-home respite while 59% recalled 
being told about the availability of out-of-home respite. In excess of one-third of the parents were 
unaware of the availability of both in-home and out-of-home respite. 



Additionally because many of the families who had been informed of the availability of respite services 
had not used them, a focus group was conducted to discuss issues related to the use of respite. Focus 
group participants included in-home and out-of-home respite providers, HBCI+ and CCM providers, 
staff from the two hospital intake sites, parents, research staff, and program personnel. In all 21 
individuals with varying perspectives and opinions concerning the use of respite services were convened 
for a three hour session to identify obstacles preventing the use of respite and to develop strategies for 
minimizing these obstacles. 



During the focus group, participants identified 37 obstacles limiting the use of respite services. 

These obstacles were then sorted into like categories by five external raters and the joint proportion 
matrix was factor analyzed resulting in ten obstacle groupings which are summarized in Table 1 . As 
shown in this table, parents and services providers' need for more information was identified as an 
obstacle limiting the use of respite. Parents expressed having many unanswered questions about respite 
and services providers indicated they were not clear about what respite entailed. 



It was also acknowledged that respite had a negative connotation and was viewed by parents and 
services providers as the first step in removing a child from the home. Parents believed that using respite 
signified they were not doing a good job taking care of their child, and services providers expressed 
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reservations about the therapeutic value of respite. 

Issues involving choice and flexibility were also identified as obstacles. Parents did not feel they were 
given a role in planning respite services and did not have an opportunity to assess the respite services. It 
was believed that children could not be matched to respite providers and that respite could not be 
accessed on an as needed basis. 

Other obstacles raised during the focus group related to the safety of children during respite care. Both 
parents and service providers wanted information about the recruitment, training, and supervision of 
respite providers. 

Return to Top 

What actions have been taken to minimize the obstacles limiting the use of respite identifled 
during the focus group? 

1 . Informational brochures have been developed. 

2. To provide complete and accurate information on the respite services, separate brochures have 
been developed detailing in-home and out-of-home respite services and are available in both 
English and Spanish. Brochures are given to families by the services provider and include 
telephone numbers of individuals who can answer any questions they may have. 

3. Respite providers have been placed on retainers. 

4. To enhance the flexibility in accessing respite, respite workers have been placed on retainer. This 
ensures that respite providers are available if and when a family/child decides to use respite. 

5. An out-of-home respite family held an "open house" for service providers. 

6. So that the service providers could meet a respite familyand become better informed about these 
services, an open house was held. 

7. In-home and out-of-home respite coordinators are informed when a new child is enrolled in the 
study. 

8. To ensure that respite is considered for each child, respite coordinators contact service providers 
when a child is enrolled in the study to discuss if and how respite services might be helpful to the 
family. 

9. Increased dialogue and awareness regarding respite. 

10. Research and program staff routinely discuss respite with service providers. Service providers are 
now asked to indicate why families did not use respite. 



What are the similarities and differences in the profiles of families and children who have used 
and not used respite services? 

Table 2 summarizes the characteristics of families and children who have used and not used respite 
services. Significant differences were found between respite users and nonusers concerning the number 
of children in the household and the identified child's age. Families using respite had more children 
living at home (mean = 3.3) compared to nonusers (mean = 2.2) and the identified child was younger 
(mean = 10.3 years old) compared to nonusers (mean =13.3 years old). Additionally, families using 
respite had significantly lower level of social support (mean = 74.1) compared to families who did not 
use respite (mean = 89.9; Barrera & Ainley, 1983). 

Although not statistically significant, families using respite services were less likely to have an adult in 
the household who was employed and the identified child was more likely to be living with neither 
biological parent. 
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No differences were found in the number of adults living in the household, the maternal caretaker's age 
or educational level, or the child's race, ethnicity, gender, degree of dangerousness, or primary diagnosis. 
Return to Tod 
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Introduction/Purpose 

Recent years have witnessed a renewed interest in emergency mobile outreach services designed to 
respond immediately and intensively to persons experiencing psychiatric crises. Various studies have 
described mobile outreach programs serving adults (Steer, Diamond, Litwok & Henry, 1979; West, 
Litwok, Oberlander & Martin, 1980), young and elderly adults (Winogrond & Mirassou, 1983), and 
families, including those with adolescents and/or young children (Bengelsdorf & Alden, 1987; Buchta, 
Range & Wetzel, 1974; Everstine, Bodin & Everstine, 1977; Kirmey, Madsen, Fleming & Haapala, 
1977). For the most part, these studies suggest that mobile emergency services are effective in resolving 
crises and deterring clients from hospitalization. This finding has, however, been called into question by 
Fisher, Geller and Wirth-Cauchon (1990). In their examination of 1986 data from Massachusetts, those 
authors found that their results "... fail to support the numerous claims regarding the ability of mobile 
crisis intervention to reduce the use of hospitalization" (p. 251). In interpreting their findings, the authors 
do not deny the value of mobile intervention, but rather, call for more rigorous evaluation in order to 
maximize its effectiveness (p. 252). 

Standing in contrast to the conclusions of Fisher, et. al. (1990), are the results of a preliminary study that 
focused upon the Children's Mobile Outreach program (CMO) offered under the auspices of the 
organization now known as TRIS Comprehensive Mental Health Services. During the program's 
infancy, the vast majority of CMO's admissions were children and adolescents whom mental health 
screeners had originally referred to TRIS for placement in its acute care program, designated as a 
Children's Crisis Intervention Services (CCIS) unit. It became apparent, however, that after receiving 
CMO services, many of these clients were stabilized and no longer needed inpatient care. In fact, an 
initial outcome study (Moore, 1990) found that, during the program's first 18 months of operation, 

68.3% of its clients were diverted from admission to more intensive acute care/hospital settings. 

The early success of this program, combined with relatively scarce and inconclusive literature regarding 
mobile mental health intervention, contributed to the decision to undertake the present study. Added to 
that was the desire to examine the program as it had evolved in response to changes within the 
environment. 



In the years following the original study, the CMO program continued to be regarded as an effective 
alternative to hospitalization for children and adolescents in psychiatric crises. As the knowledge of the 
program spread, the majority of referrals were made specifically for CMO crisis intervention services, as 
opposed to CCIS placement. The program continued to serve children and adolescents who were among 
the most seriously disturbed, however, due to circumstances arising within the broader mental health 
care system, it became necessary for CMO to incorporate a new service dimension. Subsequent to the 
opening of two new community hospital-based CCIS units (with a total of 28 beds), CMO was faced 
with declining admission rates. The program adapted by extending its services to children and 
adolescents who were being discharged from the 3 CCIS units now in the region. Retaining its primary 
goal of deterring children and adolescents from hospitalization, CMO continued to provide initial crisis 
response to children and families, and now offered adjunct, follow-up "post-discharge" services for 
clients leaving CCIS settings who required assistance in making the transition back to the community. 
Return to Ton 
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Method/Proced u res 

The present outcome study was undertaken in an effort to determine the effectiveness of the CMO 
program as modified. Clinical information was obtained from client records and/or directly from clients 
and CMO staff. All 192 clients served and discharged by the CMO program between January 1, 1992 
and December 31, 1992 were included in the study. With few exceptions, referrals to the program 
originated with local designated mental health screeners, or with the 3 CCIS units (for patients being 
discharged). Of the clients included, 131 (68%) were community referrals, and 61 (32%) were 
"post-discharge" referrals in the process of leaving CCIS/hospital settings. 

The mobile outreach team, composed of counselors, program managers and a supervising psychiatrist, 
provided clients with emergency response capability 24-hours per day, 7 days per week. Emergency 
on-site visits by a two- member team were made in response to crisis referrals and client-initiated crisis 
calls, and were supplemented by on-going follow-up contacts, again at the client's location (usually 
home). Services for post-discharge clients were initiated at the CCIS unit prior to discharge, when 
possible, and were continued at the client's residence within the commimity. 

Of the total number of clients, the majority (82%) received crisis stabilization services, including crisis 
intervention, psychosocial assessment, counseling, problem-solving, parent training, etc.. However, a 
significant portion (18%) of both community and post-discharge clients were seen for the purpose of 
"assessment only," in which case services were short-term and involved collaborative assessment, 
treatment and follow-up recommendations. 

Program effectiveness was evaluated by examining client stability, as assessed by the outreach program 
treatment team, along with residential status of all 192 clients at the time of their discharge from CMO. 
Further evaluation of program effectiveness occurred by means of a 6-month follow-up telephone 
survey. Attempts were made to contact 83 previous clients and their families (selected according to 
CMO discharge date) to determine the client's status 6 months after CMO discharge. Whenever possible, 
these telephone contacts were initiated by CMO coimselors who were known to the client and family. Of 
the 83 attempted contacts, 59 (71.1%) were actually completed. 

Return to Top 



Results and Implications 



At the time of their discharge, 67.7% of all clients served by CMO during this 12-month period were 
viewed as stable. Little difference was detected between commimity and post-discharge clients, of whom 
66.4% and 70.5%, respectively, were considered stable. However, when the 34 clients who had received 
"assessment only" services were removed from the total, the overall stabilization rate rose to 73.4 
percent. 

Discharge destinations reflected similar success. Of the 192 clients included, 72.9% were discharged to 
their homes or foster homes, 4.7% were placed in less restrictive residential settings, and 3.1% were 
discharged to alternative juvenile justice programs. Taken together, this meant that 80.7% avoided 
possible admission to an inpatient unit. 

The 6-month follow-up survey confirmed that a large percentage of clients remained stable and were 
able to continue residing in the community. At follow-up, 76.3% of clients and 67.8% of families 
perceived the client as stable at follow-up. Within the 6-month period following the completion of CMO 
services, only 2 (3.4%) of the 59 clients contacted were hospitalized and only 4 (6.8%) were admitted to 
a CCIS unit. The majority (89.8%) remained with their natural and foster families. 



The results presented here are consistent with those of the initial CMO study. Together they lend support 
to the view that short-term, intensive in-home services can have immediate impact upon children and 
adolescents who are experiencing psychiatric crises, and can thereby reduce the need for admission into 



ERIC 



3.11 



12/11/98 11:35 AM 



8th.287.htmi 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.287.html 



acute care facilities. Furthermore, these findings suggest that intensive in-home services can play a 
significant role in promoting a successful transition to the community, following hospital discharge, 
possibly reducing tendencies toward recidivism. 
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Introduction 

This paper presents data on children and adolescents moving through inpatient and residential treatment 
programs. 

The study began in 1990, and was designed to study treatment at the Menninger Children's Hospital. At 
that time, six out of seven hospital units were doing long-term treatment, which averaged one year. We 
originally designed the study to compare youngsters who received treatment as recommended with those 
whose treatment was foreshortened by financial constraints. Most treatment in this setting was supported 
by private insurance. 

By 1995, changes in insurance coverage patterns (i.e., few treatments were not foreshortened by 
financial constraints) and the development of a broader continuum of services prompted a reexamination 
of the goals of the project. We concluded that there was significant value in attempting to conduct a 
careful study of the treatment system, using standardized measures to understand and describe the 
children and families that were served. We felt it important to go beyond the superficial satisfaction and 
outcome measures required by managed care and accreditation agencies. While recognizing the potential 
for bias inherent in internal evaluation, we affirm the strengths engendered by having researchers who 
are familiar with the workings and foibles of a complex clinical system. 
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Method 

Subjects in the study were children and adolescents, ages nine to seventeen, who were admitted to 
inpatient or residential treatment at the Menninger Children's Hospital. Treatment consisted of 
comprehensive diagnostic assessment, psychosocial milieu therapy (including groups and individual 
talks), a specialized school setting, family therapy, and, in most cases, medication and/or psychotherapy. 
Youngsters and their parents completed semi-structured interviews and questionnaire measures at 
admission, at discharge from long- term (greater than two months) treatment, and at 18 and 36 months 
after admission to treatment. Treatment staff also completed behavior checklists after two weeks of 
treatment and at discharge for long-term patients. Questionnaire measures included the Achenbach Child 
Behavior Checklist and Youth Self-report as well as a number of other scales; this discussion reports 
data from the Achenbach scales at admission, discharge from long-term treatment, and 1 8 month 
follow-up (Achenbach, 1983; 1991). 
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Results 

Demographic and Descriptive Characteristics 
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For the first 303 patients included in the study, the mean age was 14.4 years old (sd = 1 .9); patients were 
about evenly divided by gender, with 48% males. Most (91%) were Caucasian, with a mix of other 
ethnic groups, none representing more than 3% of the total. Socioeconomic status, rated by the 
Hollingshead 2-factor index, was distributed as follows: I: 21%; II: 28%; III: 22%; IV: 24%; V: 3%. 

Half of these patients had no prior inpatient admissions; 16% had one prior, 19% two, 8% three, and 7% 
four or more prior inpatient admissions. The most common principal diagnoses were affective disorders 
(40%), and personality disorders (31%). Disruptive behavior disorders represented 
a significant proportion (18%), and only 5% of patients had a principal diagnosis of a psychotic disorder. 



The mean length of stay for the sample was 127 days (sd = 168); the median stay was 37 days. 
Forty-five percent of the sample stayed less than one month; 20% stayed 1-2 months, 23% stayed two 
months to one year, and 1 2% stayed longer than a year. It should be noted, due to changes in insurance 
coverage and increase in community-based options, the mean length of stay dropped from 207 days for 
the first 100 patients in the study (median = 57) to 87 days for the most recent 100 patients (median = 
27). 

At admission, patients reported moderately high levels of problems on the Youth Self-Report (mean 
Total Problems = 63.8; N = 190); parents reported more problems, with a mean T-score of 73.0 (N = 
184) for Total Problems on the Child Behavior Checklist, well into the clinical range. Unit staff, after 
observing the patients for two weeks, also reported quite high levels of problems, particularly given the 
limited time frame and contained setting (mean Total Problems = 62.8, N = 257). 
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Change at Discharge from Long-term Treatment 

Multi-variate analyses of variance were conducted to examine the change from admission to discharge 
on the Internalizing and Externalizing scales, and on Total Problem Scales, of the Achenbach checklists, 
for self, parent, and staff report. All analyses found significant decreases in problems reported. For the 
Youth Self-report (N = 46), both Internalizing and Externalizing problems decreased, but overall change 
was mostly attributable to decreases in reports of externalizing symptoms. Parents (N = 38) reported 
significant decreases attributable to changes in externalizing symptoms. Clinical staff (N = 65) also 
observed decreases in symptoms; however, the magnitude of changes they reported was somewhat less, 
perhaps because they were observing behavior in a constrained setting and a more limited time frame, 
and they reported mostly decreases in internalizing symptoms attributable to externalizing problems. 
Return toTop 



Conclusions 

In spite of multiple pragmatic and methodological obstacles, including significant organization and 
staffing changes, these initial results are encouraging. It appears that youngsters in inpatient and 
residential treatments begin treatment with elevated scores on measures of both internalizing and 
externalizing problems, and these symptoms do decrease in the course of treatment. Although they 
appear sensitive to different aspects of improvement, patient, parents, and staff all report significant 
declines in symptoms at discharge from long-term treatment, and both patients and parents also 
document this improvement at an 1 8- month follow-up point. It is notable that short and long-term 
patients did not show different rates of improvement on these measures; further analyses may separate 
those patients who have completed treatment from those who had not, and will examine other measures 
to better understand the impact of different types of treatment. 
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Introduction 

Assurance of smooth transitions to the adult service system has been identified as one of the guiding 
principals of a model system of care for emotionally disturbed children and youth (Stroul & Friedman, 
1986). Improvement in helping youth transition to the adult system was one area that Child and 
Adolescent Service System Program (CASSP) grants funded, and emphasized. However, no states have 
reported significant progress in this area to the federal agency that oversees CASSP, and many states 
identify this as an area of weakness (Judith Katz-Leavy, personal communication, Dec 1992). 

Complications to such transitions include lack of coordination between child and adult service systems 
that serve youth with serious emotional disturbance (SED), philosophical differences between the two 
systems that discourage use of needed adult services, and the common symptoms of conduct disorder 
and drug and alcohol involvement in the youth population that do not fit the criterion for chronically 
mentally ill that establish priority populations for the adult system (Stroul & Friedman, 1986). These 
barriers are assumed to cause many youth in need of services to fall through the cracks between the child 
and adult system. 

The need for transitional services comes at a particularly challenging period in the lives of youth since 
this is the period during which they are expected to finish school, move on to post-secondary school 
education/training, obtain employment, develop independence from family, and deepen friendships to 
build their social network. This is a challenging period for all youth, and youth with SED are much more 
vulnerable to the pitfalls of this age because of the nature of their disability. Thus, the lack of treatment 
is believed to have wide-ranging effects on the well-being of this population, and their likelihood of 
eventually needing more expensive services from the mental health, criminal justice, substance abuse 
and child welfare systems. 

Few researchers, however, have tackled the question by examining outcomes or service needs of 
children with SED during the years of early adulthood. This discussion presents the results of three 
outcome studies, a service utilization study and a study of model treatment programs for transitionally 
aged youth. Findings from these studies form a coherent picture of some of the characteristics and needs 
of this population, and provide potential programmatic answers. 
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Outcomes 

Transition to Adulthood; How are Youth with 
Severe Emotional Disabilities Different from 
Non-Disabled Youth? 



Mieko Kotake Smith, Ph.D. 



Method Results 
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Method 

The study sample included 41 students attending special education classes for youth with severe 
behavior and emotional disabilities in public intermediate and high schools (Severely Behaviorally 
Disabled students [SBH]), and 41 non-disabled students in the same community (Comparison Group). 
Personal interviews were used to collect data from students and their parent/guardian. Interviews were 
repeated in the 2nd and 3rd years. The student questionnaire was based on the University of South 
Florida's instrument (see Silver below), while this investigator developed the parent questionnaire. 
Return to Symposium Menu 



Results 

Characteristics of the Students: The mean age of the students with disabilities (SBH), and the Control 
group were 15.44 and 15.57 years respectively. SBH students were predominantly male (76%), while 
Control students were predominantly female (68%). SBH students were 27% African-American 
compared to only one African-American student in the Control group. All other students were 
Caucasian. Chi-square analysis was used to indicate differences between the groups. 

Living Situation: The majority of both groups (92% Control, 73% SBH) lived in a house. The majority 
of Control students (95%) lived in one place during the past 12 months, compared to only 52% of SBH 
students, suggesting a higher rate 

of residential instability for the SBH Group. Additionally, 50% of SBH students lived with one parent, 
while 61% of Control students lived with two parents. 

Family: Parent education levels in both groups were similar: 85% of the parents of the Control Group 
and 70% of those in the SBH Group had completed high school or attained higher education. Group 
differences were found on several family measures: (1) Approximately half of Control students reported 
that their family took part in recreational activities at least twice a month, while only 15% of SBH 
students reported doing so; (2) Significantly more Control students reported being able to discuss sex 
and other problems with family than SBH students; (3) Almost all Control students reported that their 
family had a religion, while only two-thirds of SBH students did so. 

Social Life: Students in both groups experi-mented with alcohol, but a significantly greater number of 
SBH students had used drugs (p < .05). Also, more Control students belonged to social or community 
groups, participated in physical activities more frequently (p < .05), and more embraced religion than 
SBH students (p < .001). Half of Control students named individuals with whom they had frequent 
contact (family, peer acquaintances) as their hero, whereas the majority of SBH students name 
individuals with whom they had no personal contact as their hero (i.e.. sports figures, entertainers). 
Family members were named as role models more frequently among Control students than SBH 
students (p < .05). Overall, SBH students showed less involvement in constructive social activities. 
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School: Students in both groups reported that at school they learned many skills, however, they reported 
acquisition of few job or parenting skills. SBH students received less guidance about getting involved in 
the community and participated in more vocational planning at school (p < .001). Both groups reported 
positive interactions with school personnel. 

Social and Daily Living Skills: Both groups were similar in their social and daily living skills, however, 
the skills level of SBH students at the third interview was similar to the level of Control students during 
the first interview, indicating that SBH students lagged behind in this development by about two years. 

Services Received: SBH students received significantly more services than Control students, including 
individual counseling, problem solving training, help in getting along with others, coordination of 
services, help in getting a job, group therapy, alcohol counseling, and drug counseling. 

Life Changes: Fourteen subjects were lost among SBH students (34% loss) during the three year 
project. Only four Control students dropped out of the study (10% loss). Four SBH student graduated 
from high school, while 13 Control students graduated. One of the SBH graduates entered college (25%) 
compared to 85% of Control student graduates. One of each group entered the military, two of SBH 
students entered the work force (50%). 

How to Promote (and Not Interfere with) 
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Effective Transition 
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Method Results 



Youth placed in psychiatric residential facilities are often overlooked in studying transition to young 
adulthood. This study examined transitional youth in both school and residential settings. 
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Method 

The Youth in Transition (YIT) study was conducted in 1990. Young adult subjects (n=215) were 
randomly selected from a pool of 359 participants, aged 18-22, who were enrolled in the National 
Adolescent and Child Treatment Study (NACTS). NACTS is a longitudinal study of children with 
emotional disturbance identified and served by the public education and mental health systems (Silver, 
Duchnowski, Kutash, Friedman, Risen, Prange, Brandenburg, & Greenbaum, 1992). In addition to 
conducting interviews with young adults, a separate interview was conducted with one of their parents 
(usually the mother; n = 156). The young adult interview instrument contained questions about 
secondary and post-secondary education and training, employment history, income, marital status, social 
involvement, and receipt of public-sector services. Parents were interviewed about factors that helped or 
impeded their child's progress in making the transition to young adulthood. 

Return to Top 

Return to Symposium Menu 



Results 
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The sample was evenly representative of youth originally served in special education classes (n=106) 
and residential mental health settings (n=109). The sample is representative of the larger NACTS 
sample. 

Service Needs 

Less than 10% (n=21) were currently in high school; a small number (n=15) were in post-secondary 
settings. High school is a major venue for receiving services, under various federal mandates. Whereas 
most of these youth received services while there were still in high school or a residential facility, this 
picture dramatically changed after they left these settings. For example, only 1 1% of the sample was 
currently receiving mental health counseling. Rates of service were similarly low for substance abuse 
treatment (3.8%) and case management (6.9%). In addition, small numbers of young adults reported they 
currently received assistance with planning a career (10.8%), getting a job (12.9%), or keeping a job 
(0.7%). 

Young adults were asked whether they would like to receive each of the above services. The most 
desired service was help with planning a career, endorsed by nearly 42% of the sample. Young adults 
also wanted help with getting ajob (34.5%), keeping ajob (23.2%), and transportation (21.1%), which 
may be related to job acquisition and retention. A desire for mental health counseling (23.9%) and help 
with raising children (24.6%) also was expressed. Parents were asked the following open-ended 
question: "What did your child need the most to make the transition to adulthood, but didn't get?" Some 
parents gave more than one answer; therefore, a total of 21 1 responses were coded. One category of 
needs was vocational assistance. Twenty-seven of the responses (12.8%) from parents fell into this 
category. Issues mentioned in this category included skills assessment, job hunting skills, and skills 
training. The data from the young adults themselves and their parents are consistent in showing the need 
for ftirther vocational assessment, planning, and on-going support. 

A second need endorsed by parents was the need for additional counseling services (38 responses, or 
18.0%). This finding dovetails with what was reported earlier on young adults' desire to receive mental 
health counseling. Other forms of assistance that parents stated their child needed, but didn't get, 
included help with staying in school (26 responses, or 12.3%), having their child learn independent 
living skills (19 responses, or 9.0%), and the need for parents themselves to get more support from the 
system (15 responses, or 7.1%). 
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Methods Results 



These studies examined the outcomes of youth discharged from a psychiatric residential program, 
homeless transition-aged youth with severe emotional impairment, and an entire county's youth who had 
received mental health services who aged-out of child services. 
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Study 1. A 5-year longitudinal study, ending 

in 1992, included 86 adolescents with serious psychiatric impairment who were consecutive discharges 
from Washington state's first psychiatric residential treatment program at Seattle Children's Home. Most 
of the participants in this study did have the type, severity, and chronicity of mental illnesses, such as 
schizophrenia and major affective disorders, which would gain them access to the adult mental health 
system. 

Study 2. A 3-year prospective study of 50 homeless adolescents living on the streets of Seattle began in 
1990. The transition-aged youth in this study had severe emotional experiences of abuse and 
out-of-home placements. The majority of participants in this study did not have the type nor chronicity 
of mental illnesses which would gain them access to the adult mental health system. 

Study 3. An evaluation of all public children's mental health services in King County, WA was 
conducted in 1992. 
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Results 



It was clear that in King Coimty, adolescents receiving mental health services were not simply a yoimger 
cross-section of the adults who receive mental health services. A large proportion of adults receiving 
mental health services generally did not receive or even need mental health services as children. To a 
large extent the adult and children's systems treated very different clientele. 

In our longitudinal study of 86 of the state's most impaired youth, by several measures of adjustment 
(e.g, number of episodes of psychiatric hospitalizations, number of arrests, number of days detained, use 
of crisis line), 19-year olds were having a hard time relative to the years before and after that year. Their 
participation in gainful activities, school and employment decreased during their nineteenth year. These 
youth were also more likely than youth reported in the 1980 US Census (BOC, 1982) to be living on 
their own (16% vs. 8%), and less likely to be living with parents or relatives (37% vs. 83% 18-19yr olds, 
and 58% of 20-24 yr olds). There has been a trend in our mainstream culture to stretch the apron strings 
during this transition time; this easement has not been extended to some of our most vulnerable citizens. 

Many of the youth in the Seattle Children's Home did meet eligibility criteria for adult services. Often 
services received were inappropriate. A number who received residential services were placed in group 
homes and hospitals crowded with older adults who were highly medicated and chronically impaired. 

For several youth, the symptom which differentiated them from their residential peers was frequent 
attempts to run away. Other youth qualified for adult outpatient case-management services which did not 
provide adequate support to help them complete developmental tasks. 



In King County, two-thirds of the 17-18 year olds receiving mental health services did not 
meet the criteria for adult services. The plight of this population is exemplified by a 17-year old 
Caucasian male with a diagnosis of conduct disorder and post-traumatic stress disorder subsequent to 
repeated episodes of physical and sexual abuse, and borderline personality traits. Raised in a family with 
a history of substance abuse, he had been in the mental health, juvenile justice, and foster care systems 
since the age of nine. At age 17 he participated in a children's intensive case management program with 
flexible funding which provided a therapeutic aide who shared his apartment, walked him to school, and 
tutored him. He received children's day treatment services with biweekly individual counseling sessions, 
weekly occupational therapy, and psychiatric evaluation, as needed. Many human and monetary 
resources have been invested in keeping this youth alive, safe, and functional. When he turns 18, 
however, these resources will no longer be available. Because he does not carry the diagnosis necessary 
for gaining access to the adult mental health system, he will not be able to receive services, he will no 
longer have a live-in aid, a counselor, a case manager, medications nor a treatment program. It is 
possible that he will spend most of his remaining years moving in and out of the correctional system. 
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So, what leads to good outcomes among this population? Using the longitudinal study data, we 
examined predictors of adjustment in young adulthood in terms of residential and financial 
independence, employment, school completion, and minimal days in hospital and arrests within each of 
three major diagnostic subgroups. We found that attaining a high level of vocational training during 
adolescence, family supportiveness of residential treatment, participation in outpatient services during 
the first six months after discharge from residential treatment, and living in a community (as opposed to 
a residential treatment or hospital setting) at six months post-discharge were related to successful 
adjustment at three months post-discharge. 

Return to Top 

Return to Symposium Menu 



Service Utilization 

Mental Health Service Utilization by Transitional Youth. 

Maryann Davis, Ph.D. 

Methods Results 



This is a study of the mental health service utilization by young adults who were treated in deep-end care 
as minors in Massachusetts. 
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Methods 

Subjects consist of 97 young adults who received treatment in adolescent day treatment, residential or 
hospital programs for public sector youth between January 1988 and December 1993. Client records 
were reviewed for all youth in these programs who were between the ages of 19 to 25 on 7/1/92. ID 
numbers were used to track all Department of Mental Health (DMH) services received between 7/1/92 
and 7/1/94 using the computerized client tracking system. The client tracking system records all services 
received by adults with case managers, and any adult receiving DMH services other than outpatient 
treatment is assigned a case manager. Group comparisons were made between subjects and a matched 
control group of other adults in the computer system (age, race, gender & geographic area matched). 
Thus, adolescents who later received adult services were compared to adolescents who did not, and to 
the average adult consumer of the same age. 
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Results 

Forty-seven percent of youth who received adolescent services received some case manage-ment as 
adults. There were no differences in the distribution of those who received services (Adult+) and those 
who did not (Adult-) across adolescent programs. (See Table If 

There were no significant group differences in primary diagnoses; with a greater percentage of the 
Adult+ group having a psychotic disorder than the Adult- group, and more of the Adult- group with 
Conduct, Adjustment and Substance Abuse disorders. The Adult+ group also averaged longer lengths of 
stay as adolescents, were older when admitted to the adolescent programs, and had more out-of-home 
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placements. Additionally, the most common adult diagnosis of the Adult+ group was Psychotic Disorder 
(40%) followed by Mood Disorder (24%) and Personality Disorder (20%). 

This is a markedly different pattern from the pattern of this group in adolescence. Of the seven 
individuals who were diagnosed as psychotic as adolescents, five (71%) were still diagnosed as 
psychotic as adults. Of the seven individuals diagnosed as mood disordered as adolescents only one 
(14%) was diagnosed as mood disordered as an adult. Three were diagnosed as having a psychotic 
disorder (43%), one with an adjustment disorder and two with other disorders. Fewer of the Adult- group 
had completed high school than the average adult consumer, but proportionately more had received 
community-based services, residential care, inpatient services and had been in psychiatric emergency 
rooms as adults. The Adult+ group had also had more encounters with case managers as adults than the 
average adult consumer. There were no group differences in marital or custody status. 
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Methods Results 



The purpose of this study was to identify the values and best practices of programs which are preparing 
and supporting transitional youth for employment, educational opportunities, and independent living. 
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Methods 

Two hundred fifty-four programs were nominated and surveyed for site visits. A preliminary survey was 
mailed to each of these sites requesting basic descriptive data on both the program and its consumers. A 
second survey was sent to the 91 programs whose descriptions indicated that they were serving 
individuals with serious emotional/behavioral disorders, and that at least a portion of these individuals 
was in the yoimg-adult age group. The second survey requesteddetailed information about conditions 
during the program's last complete reporting year. Nine sites were selected for a site visit. A team of 
professionals, including a parent advocate, visited each site. 

The information gleaned from administrators, staff, agency records, parents, and consumers permitted 
formulation of recommendations regarding program values and best practices that appear to be critical to 
the transitioning of these youths. 
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Results 

Program varied greatly in the array of supports and services available for consumers within each of the 
transition domains of employment, education, independent living, and community life. Similarly, they 
varied in the practices used and the program values that they exemplified. Nevertheless, there were 
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commonalities across all nine programs, embracing the values of: (a) consumer-centered orientation 
including consumer and family involvement, family-role resolution, acknowledgement of cultural 
differences, and building on consumer strengths; (b) community-inclusion environments with an 
emphasis on consumers functioning in community settings alongside persons without disabilities; and 
(c) continuity from consumer's perspective with a focus on consistency in staff to whom consumers 
relate and consistency across services. One limitation that most programs voiced, and which is related to 
the continuity from consumer's perspective value, is the categorical age-determined funding: in which 
federal/state policy sets arbitrary ages for admission and discharge. 

Within the domains that the programs addressed, each program attempted to meet the features of the 
comprehensiveness of supports and services value. Nevertheless, administrators, staff, and parents 
at each site identified additional supports and services that were needed for some of the consumers. 
Although all of the programs were, to some extent, providing flexibility in supports and services, a few 
of the programs retained categorical components, which were not as individualized or flexible as would 
be needed to meet consumer's needs for establishing independence. 

Regarding the safety-net of support value, one of the programs embodied the concept of unconditional 
care, and the practice of allowing consumers to identify their sources of support from the individuals 
around them. Although all of the programs shared practices related to the value of skill development 
teaching, one program stood out as having the most systematic and effective teaching methods for 
assisting consumers in acquiring community-relevant skills, particularly in the social-problem solving 
and life skills areas. This program, as well as a second, had particularly strong commitments to the value 
of outcome orientation through their systematic consumer/program evaluation and through the tracking 
of progress for each consumer across multiple goals. 
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Conclusions 



It is clear from these studies that youth with SED exiting child-serving systems face enormous 
challenges. All three outcome studies found that youth with SED, regardless of whether they were in 
special classes, or in residential psychiatric care, were less likely to finish school, gain employment and 
have adequate social support. Their ties with their families were more distant, rendering them more 
dependent on their own abilities, underscoring the need for effective transition policies that help them 
gain employment, continue training, build social supports and avoid trouble. Smith's study indicated that 
students showed positive gains from services they had received, but their development was delayed. This 
points even more strongly to the need for continued service provision to youths who age out of children's 
services. 



It is clear that many youth do not make it successfully to the other side of the services bridge; many 
youth do not receive any needed intensive mental health services, and others receive developmentally, 
culturally, and restrictively inappropriate care. It is clear that these youth are aware of their needs, with 
so many indicating desire for services. However, it is not clear how they should access them, or if they 
will be judged eligible for those services. Since policies do not support smooth transitions between 
service systems at this time, it is important to provide youth and their families with information and 
support so that they can advocate for their rights. 



It is also important for parents and all professionals working with these and other systems to begin to 
coordinate their efforts and work together for effective transition. It is clear that the knowledge is out 
there to guide better efforts in addressing these young adults' needs. Some wonderful programs, with 
values that should be carried to the transitional system, are already in place, and able to provide expertise 
to others. It is important to note, however, that only two of the nine model programs had conducted any 
kind of outcome assessment. Further research is clearly needed, not only in terms of evaluating specific 
programs, but also in evaluating the effect of policies as they are developed, and identifying 
coordinating mechanisms that will be necessary for the variety of transitional service needs that know no 
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agency boundaries. 

Evaluation of any transitional intervention will require long-term tracking, since data indicate that 
several years may be required to see long term effects. Outcomes must evaluated in terms of the domains 
of educational attainment, employment, community adjustment and quality of life. It 
is also important that the service systems be evaluated for their impact on the success of interventions. 

Some federal progress can be seen in providing services for transitional youth. Public Law 101-476, the 
Individuals with Disabilities Education Act (IDEA), requires transitional planning for all students with 
disabilities (including SED). Furthermore, this federal legislation requires that transition services must 
be included in a student's Individualized Education Plan by age 16. The act recognizes the interagency 
needs of youth with disabilities, and requires each state to formulate policies and procedures for 
developing and implementing interagency agreements which describe the role of each agency. The 
legislation also establishes procedures for resolving interagency disputes. While this legislation may 
serve as a good model, it still does not address the needs of the many transitional youth who drop out of 
school, and do not fall under the mandate of the legislation. 

States should not wait for legislation to mandate how transitional services for youth with SED must be 
handled. Most importantly, agencies should take responsibility for this population&shyp;a primary 
reason that young adults do not recieve needed services is that they fail to meet the eligibility criteria of 
existing services. This leads to the failure of services to address the developmental needs of young adults 
who have serious impairments, but are not diagnosed as having a chronic mental illness; who have more 
capacity for independent living, but need more guidance and support than individuals without emotional 
disabilities. 
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Introduction 

It has become apparent over the past ten years that the role of the family has been identified as an 
increasingly significant variable with regard to adolescent suicide. Some authors go as far as to say that 
"family related factors appear to be the most significant contributor to youth suicide" (Husain, 1990). As 
critical as these variables appear to be, very little information is available about the family characteristics 
of youths that kill themselves. Furthermore, family characteristics of youths who attempt suicide are 
often described as a side feature of investigations of other factors, and these generally focus on the 
family characteristics only after an attempt has occurred. Little is known about those family factors that 
are precursors to the suicide attempt or about the mechanisms underlying "the relation of family 
characteristics and social supports to youth suicidal behavior" (Pfeffer, 1989). 

Most studies that have attempted to elucidate underlying mechanisms have focused on constructs such as 
family communication, emeshment, cohesion, flexibility, affective involvement, affective 
responsiveness, behavior control, roles, and problem solving. Concurrent with the family research on 
adolescent suicide, a separate body of literature has developed which focuses on adolescent coping skills 
and their relation to adolescent suicide. These studies suggest that adolescent coping skills are beneficial 
in therapy situations involving suicide ideation and attempts (Orbach & Bar-Joseph, 1993). It is 
reasonable to assume that these skills are shaped, in part, by the adolescent's experiences within the 
family context. Other investigators have suggested that family therapy is an appropriate treatment 
modality for adolescents at risk for suicide (Berman & Jobes, 1992; Richman, 1986). A logical next step 
is to investigate the family's impact on adolescent coping skills and to identify potential family 
interventions focused on expanding the adolescent's coping repertoire. 

Family Literature. Adolescents often feel alienation to which the family contributes (Smith, 1981). 

Some theories suggest that poor communication patterns within the family will promote stress that 
contributes to suicidal behavior of children (Gould, 1965). Other theories suggest that lack of good 
mothering results in the child feeling abandoned and unloved (Sabbath, 1969). However, of the studies 
completed, several characteristics of families appear with consistency: family conflict (Cassorla, 1980; 
Cohen-Sandler, Berman & King, 1982; Corder, Page & Corder, 1974a; Corder, Shorr, & Corder, 1974b; 
Davidson, Choquet, & Facy, 1976; Jacobs & Teicher, 1967; Kosky, Silbum & Zubrick, 1990; McKenry, 
Tishler & Kelley, 1982; Spirito Brown, Overholser & Fritz, 1989a; Tishler, McKenry, & Morgan, 1981; 
Williams & Lyons, 1976; Withers & Kaplan, 1987); history of family suicide (Barter, Swaback & Todd, 
1968; Brent, Kolko, Allen & Brown, 1990; Cassorla, 1980; Corder, Shorr & Corder, 1974b; Davidson, 
Choquet, & Facy, 1976; parental alcohol/drug abuse (Angel, Taleghani, Choquet & Courtecuisse, 1978; 
Davidson, Choquet, & Facy, 1976; Slaby & McGuire, 1989); and family violence, especially sepal abuse 
(Husain, 1990; Knittle & Tuana, 1980; Pfeffer, 1985; Slaby & McGuire, 1989). In addition, many 
adolescents feel they are not able to communicate with their parents (Corder, Page & Corder, 1974a; 
Corder, Shorr & Corder, 1974b; Marfatia, 1975) and there is often a feeling of parental rejection or 
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disinterest (Cassorla, 1980; Davidson, Choquet, & Facy, 1976; Husain, 1990; Marfatia, 1975; McKenry, 
Tishler & Kelley, 1982). Along with this lack of perceived family cohesion, the loss of a parent or 
significant other (Barter, Swaback, & Todd, 1968; Cohen-Sandler, Berman & King, 1982; Godwin, 
1986) can often intensify the feelings of alienation and separation. 

Coping Literature. Sadowski and Kelly (1993) believe that youth who attempt suicide have "more 
maladaptive cognitive-emotional behavior response sets to problematic situations than do psychiatric 
controls and normal controls." Lower problem solving skills are associated with suicide attempts 
(Asamow, Carlson, & Guthrie, 1987; Curry, Miller, Waugh & Anderson, 1992) and Amish (1991) 
suggests that fewer reattempts result from learning and applying coping skills. One study indicated that 
social withdrawal was used more frequently by adolescents who are suicidal (Spirito, Overholser & 
Stark, 1989b). In addition, a number of authors have documented a connection between a diminished 
problem-solving capacity and suicide (Asamow, Carlson, & Guthrie, 1987; Schotte & Clum, 1987; 
Schotte & Clum, 1982). Sommers (1995) indicated that youths who have attempted suicide and those 
who frequently ideated were less involved with the family and tended to use fewer coping strategies than 
those who thought about suicide less frequently. Additionally, youth with a history of suicide attempts 
were less likely to use relaxation and more likely to vent feelings. 
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Method 

Subjects and Procedures 

The 410 participants volimteered for this project were obtained from 20-middle, jimior, and senior high 
public and private schools and five adolescent psychiatric imits in Ohio between 1992 and 1995. The 
ages of the participants ranged from 1 1 to 1 8 years. Thirty percent were male and 70 percent were 
female. Minorities represented 7% of the population. Both the participants and their parent(s) 
participated in the informed consent process. Participants completed the Family Assessment Device 
(FAD), the Suicide Ideation Questioimaire (SIQ), and the Adolescent Coping Orientation for Problem 
Experiences (A-COPE). The SIQ was used to divide participants into low and high ideator categories. 
The initial sample contained 320 youth who scored as low ideators, 71 who were categorized as high 
ideator s, and 19 who were placed in the attempter category. The FAD data were Q-factor analyzed 
separately for each group using QUANAL (VanTurben, 1975). Differences in participant coping skills 
along topologies were then identified. 
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Instruments 

All of the instruments were developed, normed, and have reading levels appropriate for the sample 
populations The validity and reliability estimates of the instruments were deemed to be acceptable. The 
SIQ consists of either 15 (ages 13 -14) or 30 (ages 15-18) items which range in specificity 
of suicidal ideation. The respondent rates each item in terms of frequency within the last month. 
Previously determined cutoff scores were used to categorize participants according to level of ideation 
and, therefore, risk (1988). The FAD was developed to describe family functioning in terms of 
transactional and systemic properties (Epstein, Baldwin, & Bishop, 1983) and consists of 60 items 
comprising seven scales which measure Problem Solving, Communication, Roles, Affective 
Responsiveness, Affective Involvement, Behavior Control, and General Functioning. The A-COPE was 
developed as a tool for measuring coping skills within the context of the Resiliency Model of Family 
Stress, Adjustment and Adaptation (Patterson & McCubbin, 1991). It consists of 54 Likert-type items 
comprising twelve scales which measure ventilating feelings, seeking diversions, developing self 
reliance and optimism, developing social support, solving family problems, avoiding problems, seeking 
spiritual support, investing in close fnends, seeking professional support, engaging in demanding 
activity, being humorous, and relaxing. 
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Results and Implications 

The low ideator sample was randomly divided into five subsamples. Each subsample's FAD data were 
Q-factor analyzed separately. The profiles from each subsample were then compared. (See Figure U A 
single stable profile was identified (N = 83). The high ideator sample was randomly divided into two 
subsamples and treated in a similar fashion, and one stable profile was identified (N = 23). Because of 
the small sample size, the attempter data were not cross validated. Again, only one profile was identified 
(N = 9). These profiles are presented below. 

The individuals belonging to the attempter profile differed most from those belonging to the high ideator 
profile in their perceptions of roles, communication, and affective involvement. Similarly, the 
perceptions of individuals belonging to the low ideator profile differed most in terms of roles, 
communications, and affective involvement. 

Coping skills as measured by the A-COPE also varied by profile. The youths in the attempter group 
were different from those in the high ideator group in that they tended to utilize problem solving (t = 
2.04, p = .05, N = 32), spiritual support (t=2.0, p = .007. N = 32), and professional support 
(t = 2.48, p = .019, N = 32) to a greater extent. Youths who fit the attempter profile differed from those 
with a low ideator profile in that they tended to used spiritual support (t = 2.26, p = .026, N = 92) and 
professional support (t = 6.17, p = .000, N = 92) to a greater extent, and were less likely to endorse 
optimism (t = 1.84, p = .07, N = 92) and avoiding problems (t = 3.24, p = .002, N = 92). Those youths 
who scored as high ideators differed from low ideators in that they tended to seek professional help (t = 
2.63, p = .050, N = 106) to a greater extent were less likely to engage in demanding activities (t=2.22, p 
= .029, N = 106), avoid problems (t = 3.92, p = .001, N = 106), solve problems (t = 2.45, p = .016, N = 
106), use self-reliance (t = 3.07, p = .003, N = 106), and seek diversions (t = 2.94, p = .004, N = 106). 

Given the limitations of these data, the following suggestions are offered subject to replication. It 
appears that adolescents from certain types of families may be at greater risk for high suicidal ideation 
and attempts, and that both the family characteristics and the adolescent's pattern of coping skills can be 
used by counselors as an indicator of risk. In addition, since the coping skills vary by profile, attention to 
shaping appropriate skills may serve both a preventive and therapeutic function. Historically, the 
teaching of coping skills was accomplished by working with the identified client. However, since these 
skills are, in large measure, learned within the context of the family, and since their use within the 
family context may serve a protective fimction, both the adolescent and the family may be appropriate 
targets for therapeutic interventions targeted on increasing the adolescent's coping skills. 
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Introduction 

Gender sensitivity and gender competence are emerging concepts in the field of children's mental health. 
Several areas of research affirm the importance of gender perspectives. First, studies of both adult 
women and men have foimd evidence of gender bias in clinicians' attitudes, diagnosis, assessment and 
treatment (Becker & Lamb, 1994; Broverman, Broverman, Clarkson, Rosenkrantz & Vogel, 1970; Ford 
& Widiger, 1989; Hoppe, 1984). 

Second, feminist researchers have pointed to the impact of gender role socialization on the development 
of adolescent females' self-esteem, body image, sexuality, and identity (Chodorow, 1978; Dinnerstein, 
1976; Gove, 1979; Gonsiorek, 1988; Pipher, 1985; 1994; and Zimmerman & West, 1975). Likewise, 
male socialization patterns have been foimd to have a dramatic effect on commimication, difficulties 
with intimacy, conflict resolution, emotional constriction and the consequences of internalized 
homophobia. 

Third, psychologists have begim to acknowledge the importance of addressing gender role socialization 
patterns in adolescence to prevent the onset and occurrence of adult mental health problems among 
women, such as depression, dissociation, eating disorders and post traumatic stress disorders (Koss, 
1990; Mowbray, Herman & Hazel, 1992; Boskind- White, 1983; Brodsky & Hare-Mustin, 1980; Foa, 
Olason & Steketee, 1987). No longer can we ignore the importance of gender perspectives in our clinical 
attitudes, assessment, diagnosis, service development and treatment of youth with mental, emotional, or 
behavioral problems. 

One initiative imder Nebraska's Child and Adolescent Service System Program (CASSP) addresses the 
need for gender sensitive and appropriate mental health and substance abuse services for youth. This 
summary describes the preliminary results and implications of a statewide needs assessment. We present 
a Gender Competency Model to provide a developmental framework for enhancing a system of care that 
is responsive to the gender-specific needs of adolescents. 
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Method 



Delbesq (1983) developed the nominal group technique to allow for idea generation while avoiding the 
problems of group dynamics. The following steps outline the process for conducting nominal groups: 



1 . Convene in a large group. Explain the purpose of the meeting, the nominal group process, and the 
role of the facilitator. 



2. Assemble participants into subgroups of no more than seven members and assign a facilitator to 



ERIC 



333 



12/11/98 12:08 PM 



8th.311 



http://lumpy.fmhi.usf.edU/CFSroot/rtc/proceeding8th/8th.3 1 1 .html 



each group. Those selected as participants should be knowledgeable about the issue and represent 
different perspectives. (We pre-assigned participants to ensure that each subgroup contained 
representatives of different perspectives.) The facilitator introduces oneself and emphasizes the 
need for full participation. 

3. Present a single question to the group and ask participants to vsrite their own responses. During 
this phase of the process, it is important that the facilitator ensures the participants proceed 
independently, without discussion, and in absolute silence. This approach allows each member to 
ponder upon his or her own ideas, to be motivated by the observance of others vsriting their 
responses, and to be involved in an atmosphere where premature decisions do not have to be 
made. 

4. Elicit individual responses in round-robin fashion until all contributions have been made. As 
responses are stated, the facilitator numbers and records them on an easel thereby ensuring each 
member's participation. During this time, the facilitator allows no discussion by group members 
regarding form, format, or meaning of a response. 

5. Guide participants to explain the meaning of their responses in round-robin fashion. The purpose 
of this clarification phase of the process is for the group to have a shared understanding of each 
response. The role of the facilitator during this step is to direct 

6. Direct the participants to select and rank order a specific number of items from the entire list of 
recorded responses. (In our study, the facilitator asked group members to select and rank seven 
responses they identified as the most important needs of female adolescents and seven identified 
as the most important male adolescent needs.) 

7. Repeat steps 3 to 6 for each question. (We repeated the process for a second question. The time for 
the entire process totaled approximately four hours). 

The Nebraska Department of Public Institutions, the State Mental Health Authority, employed a nominal 
group technique (Delbecq, 1983) to assess the gender-specific needs of adolescents receiving mental 
health and substance abuse services. The Department mailed a letter to 700 randomly selected 
professionals located throughout the state, inviting them to participate in a needs assessment. The letter 
was mailed to local, regional, and state policy makers, administrators, directors, therapists, and school 
counselors. It stated the purpose of the study and detailed the time commitment required of those 
volunteering to participate. 

Three facilitators conducted a total of five, 4-hour nominal groups at two sites&shyp;one urban 
(Lincoln) and one rural (Kearney). A total of 28 professionals from 24 organizations participated in the 
study (females =18, males = 10). Although a majority of the participants were from mental health and 
substance abuse agencies (71%), 29% represented other child serving entities including education, 
health, juvenile justice, and advocacy programs. The specific questions posed to the participants were as 
follows: 



• What are the gender-specific needs or issues of adolescents receiving mental health and substance 
abuse services? 

• What changes would you make so that mental health and substance abuse services are responsive 
to the gender-specific needs of adolescents? 



Following the nominal group process, the facilitator asked the first question regarding adolescent needs, 
elicited responses in round-robin fashion, guided participants toward clarification and asked participants 
to individually select the 7 most important needs of adolescent females. Participants repeated the 
selection and priority ranking of needs for adolescent males. The facilitator repeated the entire process 
(including prioritizing by gender) for the second question regarding recommended changes to adolescent 
services. Thus, solution generation followed identification of the needs, issues, and concerns of the 
youth. 
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Results 

The authors examined the content and the rankings of participant responses both within and across the 
five nominal groups. Similar responses were first combined into a single gender-specific item. Rankings 
across similar responses were then summed for each item. Finally, items from all five nominal groups 
were arranged into similar categorical areas and the numerical summed ranks for each item within a 
category were added together. The resulting value was then assigned to a topical category. The category 
receiving the highest summed rank represents the top priority area. This process was repeated for both 
questions. 

It should be noted that the organization of similar responses into a gender-specific item and these items 
into categories was expedited by the nominal group process. During the clarification phase of the 
process, all participants along with the facilitators, reached a shared understanding of the generated 
ideas. The authors relied upon this imderstanding as the basis for categorizing the needs and 
recommended changes into topical areas. 

Gender-specific Needs. In response to the first question, participants identified 65 unique needs, issues 
or concerns which were then aggregated into 14 topical categories. Table 1 presents the seven top 
prioritized categories for both females and males. Three issues were seen as important to both females 
and males: (1) the Lack of Adolescent Services; (2) Violence; and (3) Provider Knowledge. 

Recommendations. In response to the second question, participants generated 72 unique 
recommendations which were aggregated into five topical areas. Table 2 exhibits the five areas in which 
participants believed changes needed to be made to ensure gender appropriate service provision. The 
data clearly indicate that professionals believe that the biggest obstacles to meeting the gender-specific 
needs of adolescents are the lack of services and barriers at the macro-level. 
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Implications 

Building on the results of the present study, we have begun to identify the components of a Gender 
Competency Model for Adolescents. The Gender Competency Model can provide a framework for 
understanding the assessment results as well as a process for mobilizing service delivery systems toward 
gender sensitivity and competence. This inductively grounded model promotes the perspectives of the 
participants, and rejects any pre-existing categorization. 

The Gender Competency Model for Adolescents embraces five domains: (1) Systems Change; (2) 

Human Resource Development; (3) Commimity Relations/Public Education; (4) Service Enhancement; 
and (5) Service Development fsee Figure D. These domains suggest that steps toward gender 
competency must take place both within and outside of traditional mental health and substance abuse 
treatment systems. Participation by other systems, including Education, Public Health, Juvenile Justice, 
and Child Welfare is essential. Community involvement in and endorsement of changes are vital to the 
successful development of a gender competent system of care for our youth. Most importantly, the 
identified needs of adolescents can effectively be viewed within the framework of this model. 

Another important characteristic of the model is that there is no single approach to developing gender 
competency. For instance, the road toward competency can begin at the macro-level&shyp; Systems 
Change&shyp;by adopting legislation designed to address gender issues. These policies can in turn 
result in increased variety and flexibility in non-treatment based education (Community Relations and 
Public Education). Another approach may be to inform community members about adolescent issues and 
the need for gender competent services. Community awareness can then provide impetus for grass roots 
advocacy for increased provider knowledge and skill development (Human Resource Development). 
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Conclusion 

The nominal group process provides a highly structured, interactive way to generate ideas and solutions 
rich in information. However, it is important to recognize the limitations of this particular methodology: 

• Sample size. The group format restricts the number of participants, and participants must be 
willing and able to commit a significant block of time to the process. 

• Representation. Our sample did not reach particular subpopulations with valuable perspectives, 
such as parents of adolescents with emotional, mental and behavioral problems. In addition, 
Nebraska, a frontier state, has a heavy concentration of rural communities. Site selection reflected 
this population density. 

• Response Bias. We noted a potential recency effect from statewide trainings in the area of sexual 
perpetration. It will be important to triangulate our results with family perspectives, contemporary 
research, and survey results of current policies, services, training and personnel. 

We have argued that the concept of gender competency recognizes the role gender plays in referral, 
service delivery, differential diagnosis and clinical interventions. The Gender Competency Model is 
based on the premise that gender responsive services differ from those ignoring the gender-specific 
needs of our youth. It is developmental, for it recognizes that services, agencies and professionals 
develop gender competency over time. Gender competent services and systems are those that evolve in 
gender consciousness by recognizing the centrality of gender in the development, enhancement and 
provision of services. The next challenge is to use the Gender Competency Model framework to develop 
specific recommendations regarding systems change, service development, service enhancement, 
community relations/public education, and human resource development. 
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Background 

Current trends within the mental health field highlight a disproportionately high percentage of minority 
utilization of services as compared with minority provision of those services (McHolland, Lubin, & 
Forbes, 1990). Many minority clients do not have access to quality services. In an analysis of the types 
of mental health services Americans receive. Redick (1994) reported a decrease in the inpatient status of 
non-minorities and an increase in the inpatient status of minorities. According to the U.S. Department of 
Commerce (1994) and the 1990 Census Bureau, one third of the United States population will be made 
up of minority persons in the year 2000. 

Researchers have repeatedly reported that minority clients prefer minority counselors; seek counseling 
more often from non-European-American counselors; terminate prematurely with European- American 
counselors; perceive mental health institutions as "White Institutions," and are suspicious and/or 
resistant to treatment; and that minority professionals are more likely to render higher quality services to 
minorities because of their understanding of the customs and the language of their own minority group 
(Boyd-Franklin, 1989; Thompson & Cimbolic, 1978; Comas-Diaz & Griffith, 1988; Terrell & Terrell, 
1984). However, the impact of minority professionals is limited if their role at policy and administrative 
levels of mental health programs is overlooked. Zane, Sue, Castro, and George (1982) called for 
minority representation within the mental health field at higher than para- and practicing-professional 
levels to ensure the development of innovative services and the broadening of the minority mental health 
research base. Minority mental health training programs are one avenue toward meeting these 
challenges. 
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Program Description 

In 1987, the Florida Legislature established a culturally based mental health training program at the 
Florida Mental Health Institute (FMHI) at the University of South Florida (USF), to serve the state's 
minority population. The aim was to increase the number of minorities in the mental health professions 
and to address the mental health needs of minority communities of the state. The original bill proposed a 
training program emphasizing practical techniques applicable to mental health service delivery in 
minority communities. In response to this need, FMHI initiated the Multicultural Mental Health 
Training Program (MMHTP) in 1989. This student fellowship program emphasizes clinical/professional 
development and cultural diversity training to improve mental health service delivery in the state. Before 
the establishment of the MMHTP, FMHI started a paid internship program with Florida Agricultural & 
Mechanical University (FAMU) in Tallahassee, the state's historically black institution of higher 
education. 
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The MMHTP is unique in that it combines career development programs with training in culture-specific 
mental health service delivery. 

The program has three general objectives: 

1 . to interest minority students in mental health careers, 

2. to provide students and professionals with training that enhances their effectiveness in serving 
minority communities, and 

3. to provide professional development training that will facilitate minority advancement to positions 
of influence within Florida's system of mental health services. 

The MMHTP consists of a one semester fellowship, full-time internship placement with FMHI. Fellows 
receive a stipend, averaging $2,000 - $2,500 per semester, to assist them with living and tuition 
expenses. Fellows spend approximately 32 hours per week (i.e., four 8-hour days) in their practicum. An 
additional eight hours per week are spent participating in formal training and other activities with 
MMHTP staff. This summary reviews the fellowship training over the past five years. The training 
program has three principal components. 

Mental Health Practicum 

Field work experience is provided in residential and outpatient clinical settings in the students' chosen 
areas of interest (e.g., with children, adults, or elderly populations). FMHI, other departments at USF 
and community agencies serve as practicum sites. The fellows learn by experiencing the actual work 
environment and becoming familiar with the operations of everyday service delivery. Fellows participate 
in practicum-based internship experiences, develop valuable insights, and learn practical skills for 
m^ing career choices and facilitating career advancement. The training program allows fellows to 
explore potential career options, to test theories and practices learned in the classroom, and to develop 
skills fundamental to successful careers in the multifaceted mental health field. The fellowship training 
involves demonstration of methods plus opportunities to learn by practice. A feature of the program is 
the emphasis on marketable employment skills. Fellows must acquire specific skills, (e.g., interviewing 
clients, group techniques, clinical record keeping), and are evaluated on their performance. 

Cultural Diversity Training 

The training program is designed to provide fellows with a comprehensive learning experience in the 
field of mental health services from a multicultural perspective. The training program is intended to 
prepare ethnic minority students to become mental health professionals by concentrating of the mental 
health needs of ethnic minority and underserved populations. Through a 13-week, classroom-based 
training module focusing on self-awareness, knowledge of culture and its functions, and mental health 
issues affecting ethnic groups, fellows learn to recognize and address cultural barriers to service delivery 
effectiveness. 

Professional Development Training 

The professional development component provides fellows opportimities to learn marketable leadership, 
supervisory, and administrative skills for long-term career advancement. Fellows receive training in such 
areas as; (1) leadership skills; (2) communication skills; (3) interpersonal skills; (4) career development; 
staff management, (6) time management; (7) resume writing; and 

(8) organizational management. Professional development training includes lectures, written 
assignments, exercises and role playing. The program includes educational, practical, personal, and 
organizational instruction; program staff support fellows to help them make short and long term 
vocational decisions to best meet their interests, needs, experience, and training. Participants receive 
training to facilitate advancement to positions of influence in the mental health and human services 
fields. 

Other Activities 

The program also provides fellows, departmental and university staff with training opportunities such as 
the Cultural Diversity Workshops for Mental Health Professionals. Additionally, the MMHTP 
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collaborates with the Multicultural Child and Family Project within FMHI's Department of Child and 
Family Studies on various research and technical assistance projects (Briscoe, 1993). In these activities, 
the fellows assist in providing technical assistance and training to various community programs that 
serve ethnically diverse children and families. In this manner, students directly observe and gain 
exposure to community service and needs (Briscoe, Yang and Wright, 1993). Initially, the program 
primarily focused on working with imdergraduate fellows but has expanded to graduate levels. As a pilot 
project, the MMHTP evolved from an emphasis on speakers and lecture series to a stronger 
curriculum-based training approach. 
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Profiles of MMHTP Fellows 

A total of 35 students participated in the MMHTP fellowship program that covers 1 1 semesters from 
Fall semester 1989 through Fall semester 1993 (see Table 0. Three different colleges and universities, 
and nine different majors were represented. Four Fellows had double majors and three participated in a 
two-semester program. A total of six ethnic groups were represented in the program. Of the 35 
participants in the MMHTP fellowship program, 24 (65%) were enrolled in degree programs that require 
internships. The remaining eleven fellows were not required to complete internships to meet 
departmental requirements and, thus, participated in the program on a voluntary basis. 

Student FoIIow-up 

In the Spring of 1993, a follow-up staff survey of former fellows (N=16) showed that 10 were employed 
in mental health/human services jobs, two were completing post graduate education, two were 
completing post graduate education, two were in the process of gaining employment, one was in the 
armed services, and one was employed in a non-related job. Half these fellows had plans for returning to 
school in the mental health field, with 12.5% completing postgraduate education. 

Before entering the program, 37% of the fellows had previous work experience in mental health/human 
service fields. Overall, response to the program has been positive. The fellows credit 
the training as helping them attain mental health positions and professional recognition. As one former 
fellow commented, "the MMHTP was instrumental in getting my career off to a focused and stable start. 
I credit the program for the recognition I received as a competent clinician." 

Recommendations for expanding the program include the following: 

1 . Increase the continuum of training offered from high school through postdoctoral; 

2. Extend fellowships for additional semesters; 

3. Increase emphasis on advance research skills for students; 

4. Increase recruitment efforts and participation with other universities and colleges; 

5. Work with state and local agencies to expand the number of public service settings from 
corrections, juvenile justice, mental health 

6. and human service programs; and 

7. Expand training opportunities and information dissemination for mental health professionals in 
cultural diversity training. 

Clearly, in tomorrow's public service delivery systems and work force, the issues of cultural diversity 
will have increasing relevancy and urgency. The Multicultural Mental Health Training Program has 
demonstrated that training and leadership can increase the number of minority students who select 
mental health and human service careers. 
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Introduction/Purpose 

The Juvenile Protective Association in Chicago is currently engaged in a longitudinal study of high risk 
families. This research is part of a much broader nationwide effort to track approximately 1500 families 
for 20 years. The Consortium for Longitudinal Studies, funded by the Administration on Children, 

Youth and Families, National Center on Child Abuse and Neglect, was formed to develop and 
administer a study of families representing a continuum of risk from moderate to high (Runyan, Curtis, 
Hunter, Black & Kotch, 1995). The foci of the shared data protocols include delineating adverse 
outcomes associated with different types of child maltreatment, evaluating the continuum of 
interventions with regard to impact on children and families, and defining mediating child, parent, 
family and cultural factors for the child victim. 

The Capella Project, located at the Juvenile Protective Association in Chicago, Illinois, is a study of high 
risk families characterized by households with a history of child maltreatment, extreme poverty, infants 
prenatally exposed to alcohol and other drugs (AOD), and high exposure to neighborhood violence 
(Curtis & Schneider, in press). Unlike the other four projects that make up the consortium, the Capella 
baseline is infancy. Site-specific measures include videotaped, in-home observations of mother and 
infant social competencies during the first and fourth years. 
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Method/Procedures 



The study is a longitudinal quasi-experiment and was designed to address long-standing, methodological 
weaknesses in prior studies of the developmental consequences of child abuse and neglect (Howing, 
Wodarski, Gaudin & Kurtz, 1989; National Research Council 1993). Enhancements include an adequate 
sample size, comprehensive descriptions of subject characteristics, type and severity of maltreatment 
behaviors, type, frequency, and length of service interventions, the use of direct observations of mothers 
and infants, the use of neighborhood controls, multiple-criterion outcome measures, and long-term 
follow-up. 

Utilizing a broad-based ecological and developmental perspective of risk and protection«&;shyp;including 
infant, caregiver, and the neighborhood environment«&;shyp;it was hypothesized that both child 
maltreatment and prenatal exposure to AOD would be related to: 



1 . lower family income. 

2. lower parenting scores. 

3. higher scores of depressive symptoms and somatic complaints. 



4. fewer social supports. 
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5. perinatal risk factors such as premature birth and low birth weight. 

6. infant temperament. 

7. developmental delays in infancy. 

Other risk and protective factors not supported by the literature, but available for analysis, included 
mothers' history of victimization, family satisfaction, stressful life events, and satisfaction with the 
quality of the neighborhood. Parenting was measured using the Adult Adolescent Parenting Inventory 
(AAPI; Bavolek 1984). 

The sample consists of 300 mothers and infants. All children must be less than 18 months of age when 
they enter the study. In order to control for geography and SES, all household units are located within 
the state child protective services (CPS) Northern District boundaries for Chicago and have household 
income below federal poverty thresholds. Mothers and infants are recruited into one of three 
sub-samples: 

1 . Maltreated with Clinical Interventions: Subjects with at least one substantiated report of child 
abuse or neglect within the family household in the past twelve months. Subjects were identified 
for the study by social service agencies after referral for long-term (three to 1 8 months), relational 
based clinical interventions such as supportive counseling or psychotherapy. 

2. Maltreated with State CPS Interventions: Subjects with at least one substantiated report of child 
abuse or neglect within the family household in the past twelve months. Subjects were identified 
for the study by state CPS workers from the Northern District Office for Chicago. 

3. Non-Maltreated, No Interventions: Subjects with no substantiated reports of child abuse or neglect 
within the family household in the past twelve months. Subjects were identified for the study by 
local, community-based health and social service agencies. 
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Results 



The following analysis was conducted on 267 mothers and infants. There were significant differences 
between the three sub-samples in annual family income as seen in Table 1 . Defined as family income 
less than $5,000 per year, 33.8% of the entire sample was "super poor." The proportion of super poor 
families across sub-samples and between mothers with infants prenatally exposed to AOD compared to 
mothers whose infants were not exposed was similar. However, more mothers with infants prenatally 
exposed to AOD were living on AFDC compared to mothers whose infants were not exposed (see Table 
21 

There were no differences between maltreating mothers and neighborhood controls in any of the AAPI 
domains: inappropriate expectations of children, empathy, attitudes toward the use of corporal 
punishment, or family role reversal. No differences were discovered in the parenting scores of mothers 
with children prenatally exposed to AOD compared to mothers whose children were not. 

Neither maltreatment nor AOD predicted differences in depressive symptoms as measured by the Center 
for Epidemiological Studies - Depression Scale (Radloff 1977) or differences in somatic complaints as 
measured by the Health Opinion Survey (Macmillan 1957). However, as measured by the CAGE (King 
1986), mothers with children prenatally exposed to AOD scored significantly higher for risk of 
alcoholism compared to mothers whose infants were not exposed. 



Social supports were measured using the Duke-UNC Functional Social Support Questionnaire 
(Broadhead, Gehlbach, Degruy & Kaplan, 1988). Mothers in sub-sample 1, maltreated with clinical 
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interventions, reported fewer social supports than mothers in the CPS sub-sample or the neighborhood 
controls. AOD was not related to social supports. 

Of the entire sample, 22.6% were reported by mothers to have been bom prematurely (less than 37 
weeks gestation), 21.5% were low birth weight, and 16.3% required oxygen at birth&shyp;however 
none of these risk factors were related to maltreatment or prenatal exposure to AOD. 

Neither maltreatment nor AOD were related to infant temperament as measured by the Infant 
Characteristics Questionnaire (Bates, 1980). 

Infant development was measured using the Battelle Screener (Newborg, Stock, Wnek, Guidubaldi & 
Svinicki, 1984). Battelle Screener results include scores that represent standard deviations from the 
mean of the normal population. Of the total Capella sample, 21.1% scored at least one standard 
deviation below normal while 7% scored two standard deviation units below normal indicating serious 
developmental delays. There were no significant differences between the three sub-samples, but 
developmental delays were found in all three, including the neighborhood controls. 

Using logistic regression, of all the domains reported in this analysis, only low birth weight predicted 
risk of developmental delays. Low birth weight infants were twice as likely to be developmentally 
delayed as normal weight infants. A series of interactions were also tested, e.g. low birth weight and 
child maltreatment, low birth weight and AOD, low birth weight and premature birth, but only low birth 
weight predicted developmental delays. 

Family satisfaction, stressful life events, and satisfaction vsdth the quality of the neighborhood were not 
related to child maltreatment or prenatal exposure to AOD. 

Although history of victimization was not related to child maltreatment, mothers whose infants were 
prenatally exposed to AOD were significantly more likely to report that, as children or teenagers, they 
were either hit, slapped, shaken, or burned by a parent or someone else and that, prior to age 13, 
someone had tried or succeeded in having sexual intercourse with them, compared to mothers whose 
children were not exposed. 
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Discussion 

Although some linkages between child maltreatment and AOD have been established in the literature 
(Curtis & McCullough, 1993; Famularo, Kinscherff, & Fenton,1993), neither child maltreatment nor 
prenatal exposure to AOD were related to several perinatal risk factors, infant characteristics, or 
observable developmental delays in infancy. No relationships were discovered between AOD and 
parenting practices, maternal depression, somatic complains, or infant temperament. No interactions 
between AOD, poverty, and infant development were uncovered. 

However, several risk factors related to prenatal exposure to AOD were discovered among the mothers. 
Mothers with infants prenatally exposed to AOD were, by self-report, at greater risk of alcoholism and 
consistently reported a greater history of personal violence than mothers whose infants were not 
exposed. 
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Introduction 

The Total Family Strategy program (TFS) funded by the Juvenile Welfare Board of Pinellas County 
provides early intervention services to at-risk families in Pinellas County. The program is designed to 
prevent and ameliorate costly social problems through family empowerment, competency building, and 
counseling. 

The program is centered on an empowerment and competency based model. This model is founded on a 
belief in the inherent strengths and abilities of family members. The model focuses on enabling families 
to identify resources in their own family and the community to resolve their problems. To help families 
accomplish their goals, the TFS program offers individualized services to all members of a family. 
Services may include in-depth assessments, case management, counseling, family therapy, parent skill 
building, job training, mental health, aftercare, follow-up services, and temporary financial assistance. 

This program is provided by four agencies in Pinellas County. These agencies have the responsibility for 
organizing their program and hiring and supervising the case workers and supervisors who provide 
services. The agencies work with the Juvenile Welfare Board to ensure that staff are appropriately 
trained in the principles of family empowerment and family based service. 
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Method 

Participants 

A total of 181 families who were referred by social service agencies, schools or self-referrals were 
admitted to the program. Eligible families, as determined by a risk factor checklist, typically have low 
incomes, are usually headed by a single parent, and have histories of poor coping, domestic violence, 
alcohol, or substance abuse. Thirty-eight percent of the families served by the program received their 
primary income from AFDC, and 79% had a total income of less than $1,000 per month. Among the 388 
children served by the program, 69% were in families of two or more children with 72% of the children 
under the age of seven. Presenting problems for families in the program included defiance and problems 
at home and school on the part of the children, and lack of parenting skills on the part of the parents. 

Procedures 

The evaluation of the Total Family Strategy program (TFS) was conducted through the collection and 
analysis of qualitative and quantitative information. Case workers collected information using the 
following instruments: (a) demographic face sheet; (b) a risk factor checklist which defined the 
eligibility of the family for the TFS program; (c) a subset of the Child Well Being Scales and the Family 
Risk Scales; (d) pre and post-intervention summaries which detailed specific and unique goals, 
expectations, and accomplishments identified and defined by family members and the case workers. 

Information was also obtained from family members regarding their attitudes and expectations toward 
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the TFS program. Qualitative data were obtained from family members participating in the program 
through the use of focus groups and personal interviews. To help assess change over time, quantitative 
data were collected when the family entered the TFS program and at six month intervals. Qualitative 
interviews and focus groups were conducted at the end of the evaluation period. 

Return to Top 



Results and Implications 

Results from quantitative assessments through comparisons of pre- and post family data indicate 
improvement in broad areas of family stability and parenting. Repeated measures MANOVAs of case 
managers' ratings of family well being and t-tests of rating changes over time are provided in Table 1 
and Table 2. These results were confirmed in focus groups and case interviews with family members. 
More stable family functioning was reflected in significant improvement in the quality of life for 
families and in the security of residence. The money management skills of family members were also 
significantly improved. 

A consistent finding of qualitative data collection efforts was the degree to which the TFS program 
empowered and valued parents as part of the treatment process, provided a broad range of specific 
services to address unique family needs, gave flexible financial assistance, and was positively 
experienced by the program participants. One of the most consistent concerns identified was the negative 
impact of case worker turnover for some families, and the need for more targeted counseling services for 
children. 

The evaluation of the Total Family Strategy program emphasized multiple sources of data to test the 
appropriateness and accomplishments of the program. A wide variety of problems identified for 
families, such as housing, employment, education, parenting skills, children's behaviors, and schooling 
were confirmed in family and focus group interviews. The TFS services placed value and emphasis on 
flexible financial support, in home counseling, and a variety of assistance for parents and children. 
Results of the qualitative and quantitative research indicate the strongest effects appear to be associated 
with improvements in the fimctioning of parents as care-givers. Parents have benefited both from 
parent-skills training and related services as well as from the reassurance, support, and hope case 
workers offer. The overall effect of the program in improving the quality of the participants' lives was 
reflected in focus group and parent interviews. Parents and case workers agree that there has been a 
significant improvement in the families quality of life. 

Future evaluation and program reviews may focus specifically on issues related to program effects for 
children, long term impact on families who have graduated from active care, and a comparison of the 
effectiveness of the TFS program to alternative service interventions. 
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Introduction 



Since 1985, the Ventura Coimty system of care has served a defined group of children and adolescents 
for whom local public agencies have a mandated responsibility, viz., court wards, court dependents, and 
special education students who are at risk for out-of-home placement. Although they represent the most 
significant cost for public agencies, these youngsters represent a fraction of those with subthreshold 
problems (Costello & Shugart, 1992), who, if not served or underserved, could eventually require more 
intensive and costly services such as hospitalization and residential treatment. This paper describes work 
in progress whose long-term goal is to link local early childhood efforts with the existing system of care 
and, by doing so, broaden the target group served by the system. An important part of this effort is a 
developing partnership between the Ventura County Mental Health Department (VCMH) and Child 
Development Resources of Ventura County (CDR), administrator of the county wide Head Start 
program. This collaboration seeks to improve services to at-risk children who are younger and 
demonstrate less impairment than the current target population but who could eventually become 
involved with local public agencies. 



Local Head Start Program 

Program Structure and Services 

Organization of the local Head Start program reflects goals and service requirements set forth in federal 
performance standards (U.S. Department of Health and Human Services [DHHS], Administration for 
Children, Youth and Families, 1992; 1993). The program offers center- and home-based options. Most 
children participate in the center-based program which includes 49 part-day classes (i.e., 3.5 hours per 
day, 5 days per week) at 17 centers throughout the county. Classes are staffed by a teacher, teacher aide 
and parent volimteer(s). Average class size is 18.5. A family service specialist (FSS) is also assigned to 
each center. The average FSS caseload is 60. 

The home-based option includes five home-based groups each comprised of a teacher and 12 children. 
Teachers visit each child at home once a week for a 90-minute session and supervise two group events 
each month. Unlike the center option where a FSS provides family support, the home teacher initiates all 
family services. 

Enrolled children and families receive services in five program areas: education (regular and special 
education), health (medical, dental, and mental health), nutrition (provide part of child's daily nutritional 
needs and parent education), parent involvement (parent education and participation in program 
planning/operation), and social services (family needs assessment, referral, and community outreach). 



Population Served 

Data from the Head Start Program Information Report (PIR) for 1993-94 reveal that the local program 
enrolled 1,167 children from 1,062 families (fimded enrollment for the year was 966 children). This 
group included nearly twice as many 4-year-olds (746; 63.9%) as 3-year olds (421; 36.1%) and nearly 
the same number of males (582; 49.9%) and females (585; 50.1%). 
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The data also suggest the importance of cultural, economic, and mobility issues in serving this 
population. For example, the children were predominantly Hispanic with the proportions of enrolled 
Caucasian and Hispanic children differing markedly from the distribution of ethnicity found in the 
general county population (see Table 1). Furthermore, 774 (66.3%) of the children were enrolled in 
Medicaid/EPSDT and 1,007 (94.8%) of the families had incomes at or below federal poverty guidelines. 
Finally, the PIR shows that 258 (34.6%) of the 4-year-olds had been enrolled during the previous year 
and that 196 (16.8%) of the children enrolled in 1993-94 dropped out after classes began during the year. 
Although the number of 3-year-olds enrolled in 1992-93 and the number of 4-year-olds in the 1993-94 
program year who were enrolled in Head Start outside the local program are unknown, the data suggest 
that a number of children do not enroll for a second year. 

Data obtained on 695 families at the beginning of the current 1994-95 program year indicate the 
presence of other risk factors. These include the numbers of single-parent families (301; 44.3%), 
families in which mother (521/672; 77.5%) or father (122/385; 31.7%) is unemployed, families in which 
mother (350/670; 52.2%) or father (217/386; 56.2%) is not a high-school graduate, and families 
indicating a need for social services (884; 83.2% of family needs assessments completed). 

Special-Needs Population 

Federal performance standards (DHHS, Administration for Children, Youth and Families, 1993) specify 
that at least 1 0 percent of a program's funded enrollment be allocated to serve children with a 
professionally diagnosed disability and that such children receive services in accordance with an 
individual education plan (lEP). Of children enrolled locally in 1993-94, 112 (9.6%) had a 
professionally diagnosed primary disability. By far, speech/language impairment (73; 65.2% of 
diagnoses) was the most common disability /see Table 2). In contrast, only 3 children were diagnosed 
with emotional/behavioral disorder (2.7 % of diagnoses or 0.3% of enrolled children). 

A concern is that the small number of emotional/behavior disorder cases identified may reflect 
assessment that uses a clinical approach that is too narrowly focused and that lacks an appropriate 
developmental perspective (Fomess & Finn, 1993). Alternatively, assessment may need to focus on 
defining impairment in child fimctioning and identifying absence of family and community supports. 

Program Linkage 

Basis for Collaboration 

Based on their shared interest in early inter-vention with young children, Ventura County Mental Health 
and Child Development Resources began a partnership to serve children in Head Start. The agreement 
was approved by the County Board of Supervisors and called for VCMH to dedicate a full-time 
psychiatric social worker to serve as mental health consultant to the Head Start program for the 1994-95 
program year. Since the social worker delivers services in accordance with federal performance 
standards for meeting the mental health needs of Head Start participants, the agreement was timely given 
recent recommendations for strengthening services to enrolled children (Piotrkowski, Collins, Knitzer, & 
Robinson, 1994). Also, the agreement allowed CDR fimds that were designated for mental health 
consultation to be leveraged vvdth VCMH Medicaid billing where medical necessity could be established. 

From its perspective, VCMH is anticipating the impact of a statewide initiative to manage mental health 
services for Medicaid recipients. Under the plan, Medicaid fimding for services to the chronically ill and 
for outpatient and acute care will be consolidated at the county level. In the process, state treatment 
dollars and the responsibility for serving all Medicaid-eligible children and their families (including 
most Head Start families) will shift to counties. Since fimding will eventually be capped, a county can 
minimize the risk of exhausting its fimd by developing a managed care system that minimizes 
expensive, out-of-home services and maximizes the effectiveness of less expensive, early intervention 
services like Head Start. 



Head Start VCMH Consultant 

The social worker provides all mental health services specified in Head Start performance standards: (a) 
consults \vith teachers and family service specialists; (b) biannually observes all children in a classroom 
setting; (c) assists in providing assessment, treatment, referral and case management services to referred 
children; and (d) works \vith parents to help achieve the goals of the mental health program. Wherever 
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appropriate, service delivery is arranged to meet VCMH Medicaid billing requirements for assessment, 
treatment, and case management. 

Referred Cases 

The 3 1 referrals received by the social worker since implementation of the agreement have come from 
teachers (20; 64.5%), family specialists (7; 22.6%), public health clinics (3; 9.7%), and parents (1; 

3.2%). Three centers with a total enrollment of 156 referred no children and one center with an 
enrollment of 40 referred 4 (10%) children. More of the referred children were male (17; 54.8%) and 
most were Hispanic (21; 67.7%), 4 and 5 years-old (26; 83.9%) and Medicaid-eligible (21/30; 70.0%). 
Presenting problems most frequently identified by the referral source were emotional/behavior problems 
(12; 38.7%), parent concerns (6; 19.4%), and family problems (5; 16.1%). It is noteworthy that none of 
the referred children could be given a DSM Axis I primary diagnosis and only one could be given a 
diagnosis on Axis 2. 

Important family characteristics of referred children are reflected in the numbers of single-parent 
families (16;, 51.6%), families without a wage-earner (14/30; 46.7%), AFDC-eligible families (21/28; 
75.0%) and families previously investigated by child protective services (8/28; 28.6%). Spanish was the 
primary language in the home for 7 (22.6%) of the children and 7 (22.6%) resided in homes that were 
bilingual. 

These results raise a number of key issues. First, the referral process must be developed to insure that the 
consultant serves those children who are most in need. Important steps here include: (a) developing 
explicit guidelines for referral; (b) training staff in these guidelines; and (c) encouraging parent 
participation in the referral process so that problems in non-school settings are identified. Second, 
revenue has not been generated as expected since none of the 3 1 referred children met the medical 
necessity criteria for mental health services under Medicaid. While these children continued to receive 
services from the social worker beyond the 60-day period for determining medical necessity, an 
important element in the partnership has not been realized so far. This result indicates the need to learn 
more about the mental health and other needs of this population and to arrange service delivery in a way 
that satisfies the respective priorities of Head Start agencies and public mental health. 

Return to Top 



Implications 

Early intervention seeks to build upon the network of resources that support a child's growth at earlier 
points in the developmental process. As a practical matter, introducing early intervention in an existing 
system of care whose services constitute tertiary prevention requires a significant expansion in focus. In 
particular, criteria for a broader target population must be developed (Jordan & Hernandez, 1990) since 
few children in early childhood programs such as Head Start meet the criteria for the current target 
population (i.e., psychiatric diagnosis, significant functional impairment, and out-of-home placement 
risk). The expanded criteria may well include critical events that reflect early public agency 
involvement, age-appropriate norms for functional impairment, and the absence of key family and 
community supports. 

New outcomes and services must also be developed and integrated within the existing system. Change 
resulting from this process must be seen as a systematic extension of the deep-end of the system toward 
its shallow-end, rather than as an unrelated patch. Indeed, as change is introduced, the enhanced system 
must continue to monitor and manage outcomes for its population with SED (e.g., out-of-home 
placement) since it is the avoidance of the cost of deep-end services that assures ftmding for the early 
identification and community-based treatment. 



Achieving these goals will require a research agenda which includes various tasks: (a) develop a 
management information system that provides easily retrievable data on child and family characteristics, 
referrals made, services delivered, and outcomes obtained; (b) develop and test a developmentally 
appropriate model for identifying children with serious functional impairments; (c) develop a set of 
outcomes to be assessed annually (e.g., increase the number of children who return from the previous 
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year's enrollment and reduce the number of children who leave the program during the year); and (d) 
develop a model for transitioning children from Head Start to elementary school programs and tracking 
of their subsequent involvement with public services over time. 
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Introduction 



The purpose of the present study was twofold. The first objective was to compare three common 
measures of agreement in describing child behavior problems. Pearson correlations, Cohen's kappa, and 
percent agreement calculated from the data were compared in order to examine characteristics of each 
measure in describing interinformant agreement for ratings of child behavior. Researchers have 
calculated agreement differently thus complicating comparison across studies. Some of the putative 
measures of agreement suffer from flaws Aat are not recognized or acknowledged (e.g., Bartko & 
Carpenter, 1976), and researchers often make assumptions about the appropriateness of measures with 
little rationale (e.g.. Berry & Mielke, 1988). The blind use of an analytical tool without acknowledgment 
of its shortcomings and appropriateness for measuring the construct of agreement in a specific situation 
may produce results that are misleading as to the actual degree of agreement between informants. 

Typically, agreement has been expressed using product-moment correlations of standardized summary 
scores across pairs of informants (Achenbach, McConaughy, & Howell, 1987; Eisenstadt, McElreath, 
Eyeberg, & McNeil, 1994) or correlations across items for pairs of informants (Achenbach, 1991; 
Fischer, Barkley, Fletcher, & Smallish, 1993). Although correlations reflect relative association between 
informants, they are not appropriate indices of interinformant agreement (Bird, Gould, & Staghezza, 
1992). Correlations reflect only the association between groups of items and not absolute agreement 
between raters. It is possible to obtain a high degree of association (correlation) yet little agreement 
within the pair (Jensen, Traylor, Xenakis, & Davis, 1988). The pattern of responses within a pair of 
informants may be almost identical yet be anchored at different points on a scale. 

An alternative index of agreement is kappa, first proposed by Cohen (1960), which assesses the 
proportion of agreement after removing the effects attributable to chance. It too has been criticized. Not 
only is the statistic tedious to compute, its values vary with factors other than agreement, specifically 
sensitivity, specificity, and the prevalence or base rate of the target illness or behavior. Several 
researchers have acknowledged some of these problems (Brennan & Hays, 1992; Feingold, 1992; 
Hutchinson, 1993; Maclure & Willett, 1987) including Spitznagel and Helzer (1985) who noted that the 
problems caused difficulty in comparing kappa values across studies. Even so, researchers have not 
adequately recognized and addressed the impact of varying base rates on kappa within studies. 

Finally, an expression of interinformant agreement that avoids the problems that plague kappa and 
correlation is percent agreement. Percent agreement is relatively easy to calculate, and researchers and 
practitioners alike comprehend its meaning. Recently, more researchers investigating interinformant 
agreement of child behavior ratings have utilized percent agreement as the descriptor of choice (Kaslow, 
Warner, John, & Brown, 1992; Kolko & Kazdin, 1993; Renouf & Kovacs, 1994). Although the statistic 
does not account for chance agreement, the argument is made here that when considering agreement 
between experts' ratings, chance agreement is inconsequential. The rationale needs to presented for 
making deductions rather than vice versa. 
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The second objective of this study was to examine several child, parent, and dyadic-level variables for 
significant correlations with mother/father agreement about their child's behavior. Particular emphasis 
was placed on looking at dyadic-level variables that have received scant attention in the literature 
relative to individual-level variables such as gender and child age. 

Return to Top 



Method 



Participants were children and their families who were referred to a pediatric clinic for assessment of 
common behavioral concerns and for whom Child Behavior Checklist (CBCL; Achenbach, 1991) data 
were available from both parents. Sixty children, 46 males and 14 females, ranging in age from 4 to 12 
years (M = 6.7, SD = 2.05), comprised the sample. Ninety percent of the sample was Caucasian. All 
lived with two parents, 80% with both biological parents while the remaining children's parents were 
divorced and remarried. Finally, participants represented a wide range of socioeconomic levels. In 
addition to the CBCL, parents completed a variety of measures including the Duncan Socioeconomic 
Index (Duncan, 1961), the Mental Health Inventory (MHI; Veit & Ware, 1983), the Dyadic Adjustment 
Scale (DAS; Spanier, 1976), and the Life Experiences Survey (SLES; Sarason, Johnson, & Siegal, 
1978). 
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Results 



Measures of Agreement 

Figure 1 demonstrates that, given the same number of disagreements, kappa values, and to a lesser 
extent correlations, decreased with higher mother/father pair base rates, or more agreement that the 
behaviors did not describe their child. For children who exhibit few problem behaviors, parents are more 
likely to have relatively more response combinations in one cell (0,0 for the CBCL). Because of kappa's 
calculation and definition of chance, when parents agreed that a substantial number of items did not 
characterize the behavior of their child, a larger deduction in the pair's kappa value was made to account 
for chance, thereby reducing the observed kappa value. Thus, over pairs, there is greater variance in the 
measure resulting from factors not related to agreement. When used as a dependent measure, the 
increase within measure variance of kappa and correlations due to marginal symmetry and higher base 
rates make them far less sensitive to the independent variables. In addition to Figure 1, this fact also is 
demonstrated in the tables by fewer significant findings when using kappa or correlations as a measure 
of agreement. 

We recommend using percent agreement as a measure of agreement between respondents when they are 
providing judgments about which they can be considered an expert, and particularly when one response 
category might be more frequently used. Percent agreement is easier to calculate and understand 
conceptually than are other measures of agreement. As mentioned above, it also can be more powerful if 
other measures suffer from large within-measure variance. 

One criticism of percent agreement as a measure of agreement is that it does not account for agreement 
due to chance. However, we assert that chance has very little impact in observed agreement when 
respondents are asked to make judgments about which they can be considered experts. Parents make 
expert judgments about their children's behavior over time. Thus, the similarity between informants' 
responses results almost solely from similar perceptions rather than from chance. 

Factors Related to Mother/Father Agreement 

Average agreement between mothers and fathers was high, 73.5% (SD = .10, range 40.7% - 94.8%). 
Percent agreement correlated with several child and parent characteristics including mothers' and fathers' 
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levels of psychological distress, general well-being, and reported recent positive life events. Agreement 
also was negatively related to fathers' reported negative life events and positively related to mothers' 
reported SES. 

Dyadic psychological distress and general well-being were significantly correlated with agreement. 
Couples with more distress showed less agree-ment, and couples with higher well-being scores showed 
more agreement. Mother/father agreement on their children's behavior was negatively correlated with 
discrepancy scores for marital satisfaction, affection, consensus, and negative life events. Our findings 
emphasize the need to look at dyadic- and family-level variables in addition to individual characteristics. 
Please refer to Table 1 and Table 2 for significant correlations between agreement and individual and 
dyadic-level variables. (Although percent agreement is supported here as the index of choice, results 
using kappa and correlations are retained for illustrative purposes as well as to compare to other studies.) 
Return to Top 



Implications 



On average, we found a high degree of agree-ment between parents when reporting on their children's 
behavior. This finding is important in relation to screening. Because of the similarity in mothers' and 
fathers' reports, screening can be undertaken using the report of only one parent. However, once a child 
is identified or when a more in-depth assessment is made, information should be collected from both 
parents to provide a more complete picture of the child and family context. Additionally, our findings 
emphasize the need to look at dyadic- and family-level variables in addition to individual characteristics. 



In the future, more research should investigate the outcomes of agreement. For example, do children 
whose parents exhibit higher agreement demonstrate better outcomes? Is higher mother/father agreement 
related to treatment compliance or remaining in treatment? An interesting finding of this study was that 
the number of disagreements about the presence/absence of a behavior was more than twice the number 
of disagreements about the frequency of a behavior. Future research should investigate the hypothesis 
that better outcomes would be exhibited by children whose parents disagreed more about the frequency 
of behaviors than about the presence/absence of behaviors. 

Return to Top 
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Introduction & Method 

Intensive Case Management (ICM) for children is New York State's most intensive, home-based service 
option within its community based system of care. Intensive case managers are available to children and 
families at all times and serve a small caseload of ten children and families. They perform a 
comprehensive assessment of family and child needs and link families to needed services. Work with 
children and families is done in settings outside of the office and providers are expected to actively 
advocate for enhanced service delivery for children and families and system change. In addition, flexible 
service dollars are available for the purchase of services otherwise unavailable. 

In evaluating outcomes for children enrolled in ICM, researchers at the Bureau of Evaluation and 
Services Research of the New York State Office of Mental Health conducted a longitudinal study of 
a sample of 199 children enrolled in the program. On average, children in the sample were 1 1 years of 
age, white, non-Hispanic, mostly male, primarily lived in single parent households, were in the custody 
of a biological parent, and went to school in a special education setting. The most common diagnosis 
among these children was disruptive behavior disorder. They were out of home (either hospitalized or in 
out-of-home placement) an average of two times prior to enrollment, were functionally impaired in an 
average of 2.5 out of 5 areas, and displayed an average of 5.5 problem behaviors or symptoms (out of 
25). 

The study included a number of measures to describe the status of children and families enrolled in the 
program. To measure functioning status of both children and families, the Child and Adolescent 
Functional Assessment Scales (CAFAS; Hodges, 1990) was administered at enrollment and at three 
years or discharge. In addition to CAFAS scales, child status was also measured using a yes/no checklist 
of five areas of functional impairment, a yes/no checklist of presence of 25 behaviors and symptoms, 
and from the parent perspective, the Child Behavior Checklist (Achenbach, 1991). Additional measures 
of family status were an assessment of family problem conditions, evidence of abuse in the family's 
history, and six selected scales from the Child Well-Being Scales (Magura & Moses, 1986). The 
availability of this set of child and family functioning measures allowed for an examination of the 
validity of the CAFAS measure for this population of children with serious emotional disturbances and 
their families. 

A factor analysis of the CAFAS scales produced two discrete factors. The CAFAS subscales fell cleanly 
into a Child Functioning factor (Child CAFAS) made up of the Role Performance, Behavior Towards 
Self and Others, Moods and Emotions, and Thinking subscales and a Caregiver Resources Factor 
(Caregiver CAFAS) made up of the two caregiver resources subscales. The Substance Abuse subscale 
did not load on either factor and was omitted from this investigation. These factors along with the Total 
CAFAS Score (omitting substance abuse) comprised the key measures that are examined by the analyses 
that follow. 
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Results 



Family History of Abuse 

Figure 1 displays the ability of CAFAS scales to discriminate between families that were reported as 
having evidence of various types of abuse and those that were not. For all types of abuse or neglect, the 
Caregiver CAFAS and Total CAFAS mean scores were significantly different for groups of families 
where such abuse and neglect was present, versus those that had no such evidence. In the case of sexual 
abuse, the Child CAFAS score was able to discriminate, as well. 

Family Functioning 

On measures of family problems, caregiver capacity for childcare, primary caregiver/spouse relationship, 
continuity of caregiving, caregiver's ability to recognize the child's problem condition and caregiver's 
cooperation with service provision, the Caregiver CAFAS scale significantly discriminated between 
groups of families that were clustered by high and low levels on each measure. The Total CAFAS 
significantly discriminated between groups of families that were identified with high and low levels of 
family problems, caregiver capacity for childcare, continuity of caregiving, and caregiver cooperation. 

Areas of Functional Impairment 

A yes/no checklist assessing functional impairment in five areas was administered to intensive case 
managers when children were enrolled in the program. The Child CAFAS and Total CAFAS scores 
significantly discriminated (at various levels of significance) between children identified as being 
impaired in the areas of self-care, cognitive fimctioning, and self-direction (see Figure 2). The two areas 
where no significant discrimination was observed have heavily skewed distributions. Almost all children 
were identified as impaired in the area of social relationship functioning and very few were impaired in 
the area of motor functioning. 

In general, CAFAS measures did not dis-criminate for the presence or absence of symptoms/behaviors 
for this population, adding to evidence that measures of functioning and symptomatology may be 
independent. Only in the area of psychotic symptomatology was the Child CAFAS scale able to 
discriminate among children who displayed these symptoms from those who did not. 

The Child CAFAS scale was able to discriminate between children who scored in the clinical range on 
the parent completed CBCL Total Problem, Internalizing, Externalizing scales. The Total CAFAS only 
discriminated on the Total Problem Score (see Figure 3). 

CAFAS and ICM Outcomes 

A logistic regression analytic model was used to examine the ability of CAFAS scales to discriminate 
between children who were hospitalized after enrollment in the program and those who remained in the 
community. This model successfully classified hospitalization outcomes for 87.3% of children (see 
Table 1). Male children with high average total CAFAS scores (20 to 30), who were referred from 
mental health programs, lived in non-family settings at enrollment, and had recent contact with the 
mental health service system, had a 72% probability of being hospitalized after enrollment in ICM. This 
model identified the presence of these conditions as corresponding with the greatest likelihood of 
hospitalization. 

Shifting each measure in the model to an opposite or less severe condition, while leaving the others 
unchanged, decreased the probability of hospitalization and tested the strength of each measure's 
contribution to the prediction of a hospitalization outcome. Change in the CAFAS measure made the 
greatest impact on the prediction of hospitalization. A child with a low average CAFAS score (0 to 10) 
had a 32% probability of hospitalization when all the other measures were unchanged, a decrease in 
probability of 41% from a high CAFAS score. Shifting from a High CAFAS to a Medium CAFAS (10 
to 20) reduced the probability only by 8%. 
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Conclusion 



From the perspective of the evaluation of ICM, the correlation between the CAFAS measures and CBCL 
scores suggest a shared sense of children's problem severity among parents and providers. More 
generally, the data presented here contribute to the body of evidence that the CAFAS measure is a 
psychometrically valid measure of child and family functioning. Its attractiveness as a measure is further 
strengthened by the ease with which the scales can be successfolly administered to providers and the 
resultant high response rate. 

Return to Top 



References 

Achenbach, T.M. (1991). Manual for the child behavior checklist and 1991 profile. Burlington, VT: 
University of Vermont Department of Psychiatry. 

Hodges, K., (1990). Child and adolescent functional assessment scales. Nashville TN: Vanderbilt Child 
Mental Health Services Evaluation Project. 

Magura, S. & Moses, B. S. (1986). Outcome measures for child welfare services. Washington, D.C. , 
Child Welfare League of America 
Return to l op 



Steven Huz, MPA 
Research Scientist II 
518/473-9559 Fax: 518/474-7361 
E-mail: SRHUZ@NKI.IRIS.RFMH.ORG 

Mary E. Evans, Ph.D. 

Interim Director 

518/474-7209 Fax: 518/474-7361 
E-mail: SREVANS@NKI.IRIS.RFMH.ORG 

Thomas L. McNulty, M.A. 

Research Scientist I 

518/473-6002 Fax: 518/474-7361 

E-mail: TLM238@UAL562.ALBANY.EDU 

Bureau of Evaluation and 
Services Research 
New York State Office of 
Mental Health 
44 Holland Avenue 
Albany, NY 12229 

Return to Top 



ERIC 



381 



12/11/98 1:40 PM 



gth ^40fi http://lumpy.tmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.349fl .html 



CO 

CO 

CO 




yj 





SL. 

o 

o 

yj 

uSl 



:RiC 

BS&m 







lofl 12/11/98 1:03 PM 



8th.349t2 http://lumpy.tmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.349t2.html 




lofl 12/11/98 1:04 PM 



ON 

rf 

cn 

•S 

oo 

OO 

W) 

c 

0> 

0> 

o 

o 



a. 

o 



oo 

Os 



<N 



U 

t: 

o 

S 

c/5 

pu, 

u 

■T 3 

a> 



CD 

00 

00 



3 

2 



o. 

G 

3 



& 







yj 

=Ji 



cg5E 



©=. 



O 

m 

OJJ 



CO 

CO 

CO 



^mc 



00 



8th.355 



http://lumpy.fhihi.usf.edu/CFSroot/rtc/proceeding8th/8th.355.html 



IHomei ibiearcni iinaexi 

8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

The CAFAS as an Outcome Measure in the Alternatives to 
Residential Treatment Study 



Author 

Introduction Method Results References 
Back to Table of Contents 



Introduction 

Psychiatric nomenclature advances a multiaxial approach to psychiatric assessment emphasizing 
measurement of five dimensions: psychiatric symptoms, personality disorders, physical health problems, 
type and severity of life stressors, and global assessment of functioning. This last dimension has evaded 
sophisticated measurement; however, development of tools to measure global functioning is increasingly 
important as funding agencies include functional impairment in definitions of youths' serious emotional 
disturbance, and third party payers rely on severity of impairment in order to qualify persons for 
intensive services (Hodges & Gust, in press). Over the past several years, Kay Hodges has been 
developing and refining the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 1990; 
Hodges, Bickman, & Kurtz, 1991; Hodges & Gust, in press). The 1990 version of the CAFAS was 
designed to measure impairments in youths' performance of expected roles within the family, school, 
and community. The purpose of the present study was to evaluate internal consistency and validity of the 
1990 CAFAS as well as its sensitivity to change over time. 
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Method 

Procedure 

The Alternatives to Residential Treatment Study (ARTS) was designed to describe five exemplary 
community-based alternatives to residential treatment programs 1 (detail in Duchnowski, Johnson, Hall, 
Kutash, & Friedman, 1993). All youth entering selected programs were eligible for study recruitment if 
they were between 6 and 1 8 years of age, it was their first entry into the program, and they and their 
caregivers consented within one month of entry. Demographic information, history of services received, 
multi-dimensional psychosocial functioning, and emotional/behavioral problems were assessed. Entry 
into the study included baseline, 6 month, and 1 year follow-up assessments. One hundred sixty- three 
youth participated in the first assessment. Youth who had complete data for baseline and 12-month 
follow up were included in the present study (130 youth; sample attrition from initial assessment to 1 
year follow-up 20.2%). There were no differences on age, race, sex, or symptom severity between those 
with complete longitudinal data and those without complete data. 

Participants 

Youth averaged 14.1 years (SD = 3.1 years). Sixty-six percent were male, 64.6% Caucasian, 12.3% 
African American, 12.3% Native American/ Alaskan, 10.0% Latino, and one youth was Asian. At the 
point of entry into the programs, these youth presented emotional, behavioral, and social problems that 
were severe, long term, and complex (Cascardi, Kutash, & Duchnowski, 1994). Youth had received 
prior residential mental health services an average of 4 times, 61% of the sample had prior involvement 
with law enforcement and juvenile justice systems, and nearly 80% percent had received Special 
Education services prior to program entry. Sixty-three percent entered the programs from an 
out-of-home residential setting. 
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Measures 

Baseline and 1-year follow-up data from the CAFAS, Child Behavior Checklist (CBCL; Achenbach, 
1991), and Rosenberg's (1989) brief 10-item Self-Esteem Scale, and type of services used over 1 year 
were analyzed. The CBCL is a widely-used measure utilizing parent report of presence and severity of a 
variety of emotional and behavioral problems of childhood and adolescence. It yields total, internalizing, 
and externalizing behavior problem scores. The Self-Esteem Scale is a youth report of general self 
worth. The 1990 CAFAS is a brief, multi-dimensional measure of impairment in functioning, with 5 
subscales each utilizing four rating categories. 
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Results & Discussion 

Reliability analysis revealed the first 5 subscales of the CAFAS were internally consistent, with 
a Chronbach's alpha of .7122. Table 1 presents information on validity of the CAFAS. The bivariate 
association of CAFAS total and subscales scores, CBCL, and Self-Esteem were evaluated with Pearson 
product-moment correlations. 

The CAFAS total score and subscales were significantly associated with CBCL scores. Deficits in 
youths' ability to conform their behavior to appropriate social roles was more significantly associated 
with externalizing than internalizing behavior problems (z = 2.51, p<.05). The magnitude of the 
correlations between the CAFAS and CBCL are moderate, suggesting each measures a different 
construct. Unexpectedly, the CAFAS and Self-Esteem measure were not significantly correlated. 
Sensitivity to change was measured using repeated measures analysis of variance and matched pair 
t-tests. There was an overall time effect for CAFAS subscales, Wilk's lambda (5,125) = 7.94, p<.001. 
Means, standard deviations, and significance tests are presented in Table 2. 

The CAFAS Total, Role Performance, Thinking, Emotional Health, and Behavior toward Self/Others all 
showed statistically significant improvement from youths' entry into the program to one year post-entry. 
Youth in this sample also showed statistically significant improvement on the CBCL, t (126) = 4.03, 
p<.001, baseline = 71.80 (SD = 8.18), one year follow-up = 68.32 (SD = 10.40). Although statistically 
significant decreases were found on the CBCL, youth, on average, did not demonstrate clinically 
significant change on this scale. Importantly, the change in mean scores on the CAFAS suggests that 
youth demonstrated clinically meaningful change, moving from moderate toward mild impairment. 
Scores on Substance Use subscales did not show significant change. This was due most likely to the 
limited impairment in this area for the current sample. 

While mean change over time suggests promise for the CAFAS's sensitivity, mean scores do not identify 
youth who may have shown deterioration or lack of change in functioning over time. The scaling of the 
CAFAS lends itself to categorical analysis so that youth showing improvement, no change, or 
deterioration can be identified. Such analysis indicated that 62% of youth moved to a "less impaired" 
category of impairment, 1 1% showed no improvement, and 27% further deteriorat-ed over time. Percent 
improvement provides additional support for the sensitivity of the CAFAS to change over time for both 
improvement and deterioration of functioning. 

What factors account for change in CAFAS scores? It is possible that the observed change was due to 
regression to the mean, maturation, or effective provision of service. Since ARTS is not a contrasted 
group design, it is difficult to rule out maturational and regression to the mean effects. However, one can 
evaluate whether change (improvement versus deterioration) bears any statistical relationship to service 
utilization. 

Caregivers were asked whether any of 50 services, from the major service sectors as well 
as paraprofessional support, were received at any time in the first year of treatment. These 50 services 
variables were collapsed into broad-ranging categories and dummy coded as: received&shyp; yes or no. 
Point biserial bivariate correlations were computed between CAFAS at follow-up and service domain, 
partialling out baseline CAFAS score. There was a significant association between improvement in 
Substance Use and probation services (r = -.21, p<.05) and juvenile justice residential services (r = -.24, 
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p<.05). This association likely is due to restricted opportunity for these youth to use illicit substances 
while under close supervision. A significant association also was found between deterioration in overall 
functioning and receipt traditional mental health residential services (including residential treatment 
centers, group homes, psychiatric hospital; r = .28, p<.05). 

One might interpret this association several ways. It is possible that mental health residential services 
were not effective. Alternatively, the most severely impaired youth may have been offered these services 
making change more difficult to achieve. 

Findings from this study suggest that the CAFAS shows promise as a sensitive outcome measure in a 
sample of youth with serious emotional disturbance. While it is difficult to isolate the reason for change 
on the CAFAS in the present study because no comparison groups were used, these findings support 
continued evaluation of this instrument. The statistically and clinically meaningful change observed in 
this sample is especially encouraging in light of studies that 

have shown that youth with serious emotional disturbance have relatively poor outcomes (Silver et al., 
1992; Wagner, D'Amico, Marder, Newman, & Blackorby, 1991). 
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Introduction 

The aims of this summary are as follows: 

• to describe an iimovative methodology for assessing children's mental health services need; and 

• to describe how this methodology can be applied in documenting the extent and determinants of 
unmet mental health services need of a sample of children receiving services in five major service 
systems. 

• To document extent and determinants of unmet need for mental health services in a sample of 
children in five major service systems as well as in the community. 

A large proportion of children with serious emotional disorder do not receive treatment. There is a dearth 
of data regarding the mental health services needs of these children (Silver, 1990). This hinders efficient, 
efficacious, and equitable mental health services provision. 

These factors were crucial in providing part of the rationale for the SED Study. A unique feature of this 
study is the study of children with serious emotional disturbance across five service settings as well as in 
a community probability sample. Selected aspects of the study methodology that are relevant for the 
assessment of mental health services need are provided below. 
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Method 

Sample 

A stratified random sample of children aged 9 to 17 years and their parent or caregiver was selected. The 
total sample size was 1,260; of these, 510 were drawn from a community probability sample and 750 
from five service systems (juvenile justice, child welfare, special education, substance use, and mental 
health). 

Measures 

The Diagnostic Interview Schedule for Children (DISC 2.3) Service Use and Risk Factors (SURF), was 
used to document aspects of the following domains; 

• demographics 

• functional impairment 
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• medical history 

• scholastic information 

• family environment 

• family management practices 

• family history of psychiatric disorder 

• pubertal status 

• verbal ability 

• instrumental competence 

• social competence 

• self-perceived competence 

• parental discord 

• service utilization and barriers (including attitudes to mental health use) 

Before there can be progress in documenting the extent and determinants of mental health services need 
of this population, it is necessary to have a methodology for describing mental health services need. 

The "Traditional" Approach 

The "traditional" approach in attempting to operationalize or quantify the need for psychiatric services in 
a service area starts with estimating the prevalence of specific disorders and associated impairment from 
epidemiological studies based on samples of children drawn from the general population. These 
prevalence rates are then extrapolated to produce estimates of the number of children in a service area 
that would be expected to meet diagnostic criteria for specific disorders. 

Disadvantages of The "Traditional" Approach Using this approach, it is not possible to relate the clinical 
and contextual characteristics of the child and family to the frequency, intensity, and types of services 
needed. It is thus not possible to 

• assess the extent to which psychiatric need is met through non-psychiatric systems (this is 
particularly relevant for the SED study since any mental health services received by the subjects 
are received in five different service systems); 

• apply the findings to various geographical areas since services may not be organized in a uniform 
manner; 

• design new types of service delivery in an efficient manner (Brewin, Wing, Mangen , Brugha, & 
MacCarthy, 1987); 

• link the assessment procedure with a framework for knowing what action should be taken when a 
particular problem is present (Brewin et al., 1987); and 

• systematically assess the degree to which individual and aggregate service needs are met. 

The Proposed Solution 

The proposed solution to the above problems in terms of assessing psychiatric need builds on the work 
of the Medical Research Council (UK; Brewin et al., 1987) and the New York State Office of Mental 
Health (Grosser, 1991). The central feature of this solution is to describe mental health services in terms 
of the types of intervention needed (Bebbington, 1990). 
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Types of intervention refers to service elements that are generalizable and comprehensible across 
different organizational units and systems of care. Examples of possible types of intervention are as 
provided in Table 1. 

Application to the SED Study 

A list of types of intervention and their definitions will be compiled. 

A list will be derived from the clinical experience of the authors, discussions with colleagues, and 
literature reviews. To estimate inter-rater reliability, 30 subjects in each system will each be rated in 
terms of their need for each type of intervention by two service providers. Each of these service provides 
will rate these 30 subjects one month later to establish test-retest reliability. The reliabilities will be 
reported as Kappa coefficients. 

Assessments of psychiatric need will be made for each child by psychiatric clinicians using data 
obtained from the SED study. 

The clinicians will make their assessments on the basis of their clinical experience, published studies 
regarding the efficacy of particular treatment modalities, and practice parameters developed by the 
American Academy of Child and Adolescent Psychiatry (Coppans, Jaffe, & Grimes, 1994). 

Prior to the assessment of the whole sample, the inter-rater reliability and test-retest reliability of the 
clinicians' use of the instrument will be established and reported as Kappa coefficients. This will be done 
by having two clinicians make judgments on 100 subjects and having one clinician make repeat 
judgments approximately one month apart on 30 subjects. 

The types of intervention received by each child will be reported by the service providers (using the 
instrument described above). 

The extent of unmet need will be calculated. 

This will be done by ascertaining which of the needs as assessed by the psychiatric clinicians are not 
being provided by the relevant service system. 

The determinants of unmet need level will be ascertained using logistic regression. 

The dependent variable will be whether a specific service was provided and the independent variables 
would comprise the individual, family, and service system characteristics. 
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Conclusions 

In this summary, we described a novel approach to the challenge of describing unmet mental health 
services need in children and we indicated how this approach would be applied in the analysis if an 
ongoing project. The data produced will be of greater use to mental health service planners than that 
available from existing methodologies. 
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Introduction 

Treatment centers face new challenges in providing services 

for children with severe emotional disturbances (SED). Given limited dollars for human services and the 
advent of managed care initiatives, they must carefully track the services each child receives, and be 
accountable for the quality and effectiveness of treatment. Too often there are gaps in essential 
information about children, interventions and outcomes. Moreover, children with SED access a variety 
of providers, both public and private. Each center has its own system for documenting care goals and 
services, but systematic communication between centers, and even within centers, is often inadequate. 

Although technology is available to help with client information needs, implementation and use of client 
information systems within human service organizations has not proven easy (see Bronson, Pelz & 
Trzcinski, 1988; Cross, Gardner & Friedman, 1993; Hallfors, Cross & Roan, 1993). 

Hastily designed systems or "canned" software packages do not necessarily provide the information 
agencies require and are often not readily adaptable to their particular information needs. Data gathering 
requirements imposed from higher administrative levels can alienate busy staff, especially if the 
information appears irrelevant to practice or redundant. Client Information Systems (CISs) are often 
implemented without adequate staff instruction in maintaining and using data. Lack of standardization 
impedes the flow of information and makes it impossible to track children across agencies for more 
powerful analyses of outcomes. 

This summary describes the first phases of development of a simple, flexible computerized Client 
Information Systems (CIS) for agencies who provide services to children with SED. The CIS will enable 
individual agencies to initiate a carefully developed core data system that they could further refine to 
meet their individual specification. It will also allow public agencies to collect standardized data about 
children across multiple systems and limit redundancy by involving the many users of data in choosing 
key elements. Providers can access information across organizational and professional boundaries to 
improve treatment. We envision a consortium of centers across the Northeast participating in large 
evaluation studies of treatment outcome and cost-effectiveness to improve service delivery. 

Our primary objective has been to create a Common Data Set structure suitable for use by any center 
providing day treatment or residential services to children. The database structure will consist of a set of 
tables, a data dictionary describing relationships among the tables, and written documentation on 
operation. We are developing this core structure in a one year project, along with a plan for field testing 
and refinement in the following year. 

Developing a CIS that can be used by many different organizations requires overcoming a number of 
obstacles. Hardware and software incompatibilities are often present, and some agencies have a strong 
commitment to a particular operating system (e.g. DOS, Macintosh, Windows). Developing software for 
one operating system is time consuming, and m^ing that software available on a number of systems 
significantly increases the complexity of the process. Second, agencies differ considerably in how data 
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flows from one part of the organization to another, and what types of data are important for a client 
database. Considerable discussion needs to take place to discover what data users need and to establish 
consensus about what core data should be standard. The Brandeis' staffs experiences with a national 
demonstration project highlight some of the problems involved in implementing a common CIS 
software package among several organizations (see Cross, Gardner & Friedman, 1993; Hallfors, Cross, 

& Roan, 1993). 

We are suggesting a minimalist approach to the problem outlined above. Our job is to ease the transition 
from a paper-based system to an electronic CIS for participating organizations. 

By providing a comprehensive, core Common Data Set (CDS) structure, we will save time and money 
for those agencies attempting to develop their own systems, and improve their chances 
of success. A standardized CIS will allow easy access to evaluation data across agencies, yet each 
organization will have maximum flexibility in using the hardware and software of its own choice. 

In addition, researchers and the consultant are working with key personnel in the public sector, who have 
responsibility for servicing children with SED. Leaders from Massachusetts government departments 
such as public education, child welfare, children's mental health, and Medicaid have been invited to 
share their information needs and perceived trends in reporting requirements. 
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Initial Steps 

Data from Phase I of the project were drawn from five eastern Massachusetts residential treatment 
centers who serve latency age and adolescent children. All of the agencies utilize some form of 
computerized client record system representing a variety of hardware and software platforms. The 
information systems are home grown and have been through several iterations. Most were created in the 
late 1980's. The organizations are medium sized with annual budgets ranging from 4.5 to 12 million 
dollars. 



Phase I of the project focused on determining which of the hundreds of data elements used by residential 
treatment centers (RTCs) participating in the project should be candidates for inclusion in the common 
data set. We began by collecting and analyzing all the relevant computerized data elements that were 
tracked by the different programs. 

It proved somewhat more difficult than expected to gather the data elements from each of these busy 
agencies. Each differed in their understanding of the project and in their ability to produce the requested 
information. With perseverance, a reasonably comprehensive picture of the elements being tracked at the 
various agencies was developed. 

To this picture, data elements from the Adoption and Foster Care Analysis and Reporting Systems 
(AFCARS) published in Federal Register (Vol. 58 No. 244) were added. AFCARS identifies data 
elements that states will be required to report to the federal government in an initiative to produce 
national information on foster care and adoption. These elements were added to determine the extent 
which the final common data set would be AFCARS compliant. However, the addition was not as useful 
as originally hoped, given that the AFCARS elements represent primarily derived data or the results of 
initial data analysis. In addition, a core dataset identified by the American Association for Partial 
Hospitalization (AAPH) Outcomes Measurement Protocol has been entered into the application. The 
core data set represents a collection of instruments including outcome measures. At the time of this 
report, these data elements have not been analyzed. 



The elements provided by treatment centers were entered into a comparison application. Elements were 
given a common element name which captured its intended function (see Table 1). Elements were later 
reviewed and grouped into three categories: core data set, administrative, and other. 

To be included in the core data set, elements needed to address the following key questions: What kind 
of children are served? What kind of services do they receive? When do they enter and leave the 
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program? Who pays for their care? How do they do after they leave? 

As shovm in Table 1, only a few elements currently collected can answer these questions. Most of the 
elements respond to administrative or management requirements. We plan to share these elements with 
RTCs for their consideration, but would not recommend them for a core data set. For the CDS, we 
pulled what we could from the list in Table 1, and then considered additional variables that could answer 
our questions. 

Table 2 displays all of the elements being considered for the CDS. Two items deserve special attention. 
For example, we are developing a unique identifier that, while protecting children's confidentiality in 
analysis and reporting, will allow them to be tracked throughout their stay, as well tracking returns to 
this RTC or any other. Ideally, we will develop the ability to track children throughout the health and 
welfare service system. The identifier may include markers that would allow family members to be 
recognized in some way. Services to siblings or other family members then could be analyzed without 
divulging confidentiality. Social security numbers are currently used to validate the right record for the 
right child. Social security numbers mask a child's identity to some degree, but require additional 
security to protect confidentiality. 

We are also paying special attention to the outcomes variables. The ones listed in this paper have caused 
us to ask additional questions. For example, how will the outcome variables be used? Will RTCs be 
compared for their outcomes? Will attention to outcomes make RTCs avoid the most problematic 
children? Measuring outcomes has become quite fashionable, but each outcome indicator must be 
considered carefully for the conclusions it implies. 
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The Next Steps 

During the next phase of the project, we will develop an advisory committee to review all data elements 
and make recommendations for further changes in the core data elements. The advisory committee will 
consist of representatives from the RTCs, state agencies, consumers and researchers. All elements will 
be subject to discussion and debate over inclusion in the CDS. Those variables that finally are included 
will then be defined and operationalized for programming in a relational data base. The data base will be 
piloted in six RTCs and further refined by the advisory committee as needed. We envision this to take 
place over a two-year period, at which point we will make the data set available for a much wider circle 
of RTCs. 
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Introduction/Purpose 

Social validity relates to the idea that behavioral interventions and outcomes must be validated by 
individuals and communities being served. Social validation research, and the related, but more public 
policy oriented. Community Concerns Report Method represent a simple, direct methodology for asking 
customers what their needs are and how satisfied they are with current services (Schriner & Fawcett, 
1988). This approach evolves from a body of behavioral research on social validation that first made the 
case for subjective measurement of behavioral interventions (Kazdin, 1977; Wolf, 1978). 

As the scope of social validation research broadens to include public policy issues, questions remain 
regarding reliability and validity (Fawcett, 1991; Storey, 1992; Storey & Homer, 1991). The Southern 
Consortium for Children (SCC) Survey of Child and Family Service Issues builds upon the work of the 
Pennsylvania Project Social Validation Survey (VanDenBerg, Beck, & Howarth, 1992), by adapting a 
previously used survey instrument for field testing in Appalachian Ohio. 

The purpose of the study was to apply appropriate psychometric tests to data already collected by the 
Pennsylvania Survey, to alter items where indicated, and to pilot the resulting instrument in four Ohio 
counties in an attempt to evaluate reliability and validity. The resulting instrument is available for use 
elsewhere, encouraging more uniform methodology in further social validation research. The study was 
carried out in two distinct phases. 
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Methodology and Findings, Phase One 



Overview of Pennsylvania Project Data 

The Permsylvania Survey was designed to obtain some measure of the opinions of stakeholders in 
Permsylvania's children's services system. Specifically, this survey asked stakeholders to rank the level 
of importance of selected children's issues and their satisfaction with how these issues were addressed. 
The goal was to identify issues of both high importance and low satisfaction, and to target those issues 
for the development of program outcome measures. 

Seven hundred Pennsylvania stakeholders were asked a series of questions regarding the importance and 
satisfaction levels associated with various children's issues. Children's issues were categorized into the 
following life domain areas: educational, vocational, safety, living arrangements, family life, emotional 
and psychological, medical and psychiatric, social and recreational, cultural, spiritual, and legal issues. 
Respondents chose 

from among values on a Likert type scale for both importance and satisfaction levels on 49 questions, 
effectively yielding 98 separate survey items. For all questions, the choices were 1, 2, 3, 4, and 5, with 1 
being the least important or least satisfied and 5 being the most important or most satisfied. 



Ten specific groups of individuals were identified as stakeholders of children's services. The ten 
identified groups were: children age 12 or older, parents of children younger than 12, parents of children 
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age 12 or older, community agency workers, board members of agencies, school board members, 
juvenile court judges, local legislators, informal community leaders, and poverty-level representatives. 

Phase One Procedures 

In phase one of the study, data collected in the Pennsylvania Survey were secured so that an item 
analysis could be performed in an attempt to obtain a meaningful measure of instrument reliability. 
Specifically, internal consistency was measured using Cronbach's (1951) coefficient alpha, which is a 
commonly accepted formula for assessing the reliability of a measurement scale with multi-point items. 
Survey items were split into life domain areas and further along the dimensions of importance and 
satisfaction, yielding 22 separate sets of data. Within each data set, a coefficient alpha value was 
determined for each set when one item was deleted, one item at a time. This allowed for the 
identification of items that significantly eroded internal consistency. 

Further statistical analysis was carried out to assess the significance of reported variability between child 
clients and all other respondents, and between groups closest to services and those groups one step 
removed from services. An independent groups t-test (Jaccard & Becker, 1990) was performed 
comparing the mean ratings for each of the comparison groups described above. 

Phase One Findings 

Given the fact that the survey instrument is in an early stage of development and there were a small 
number of items in each life domain data set, alpha values of .6 across the dimension of importance and 
.7 across the dimension of satisfaction were considered to be adequate indicators of instrument 
reliability. This was found to be the case in 1 5 of the 22 data sets, which was considered to be evidence 
in support of reliability. The instrument was found to be reasonably reliable, in general. Life domain 
areas that did not meet the standard across both the dimensions of satisfaction and importance were 
selected for alteration, while those that exceeded the standard were considered to be reliable and were 
included in the instrument for field testing in Ohio without changes. The Peimsylvania Project staff also 
suggested two minor changes based on previous experience. In all, at least one item within 8 of the 1 1 
life domain areas was selected for alteration in phase two of the study. 

It was intuitively felt that an instrument of reasonable reliability would be able to discriminate between 
child and non-child respondent groups, and between those closest to services and respondent groups one 
step removed from services. A positive finding in either or both instances was to be considered evidence 
in support of construct validity. 

When independent groups t-tests were performed on both comparison groups, results were significant. 
Statistically significant differences were discovered in 29 of 44 data 

sets, overall. The direction of significant differentiation was very consistent between respondent groups 
across life domain areas. Evidence in support of construct validity was found to be present in phase one 
of the study. 

Return to Top 



Methodology and Findings, Phase Two 



Phase Two Procedures 

Stakeholders of child and family services were selected for participation in phase two of the study in 
Athens, Jackson, Washington and Lawrence Counties in rural, southeastern Ohio (N = 192). Stakeholder 
groups were the same as the Peimsylvania Project Survey with the exception of the poverty level 
representatives, who were excluded at the suggestion of Pennsylvania Project research staff. 



It was decided that when at least 60 valid responses were received, the statistical analyses described in 
phase one would be repeated and a follow-up interview would be conducted with 10 individuals. 
Interview candidates were nominated by mental health agency workers and represented all stakeholder 
groups, with the exception of judges. Independent groups t-tests were performed only where significant 
differences between groups were discovered in phase one of the study. 



Phase Two Findings 
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The minimum standards for values for Cronbach's alpha established in phase one of the study were 
exceeded in all but 2 of 22 life domain data sets in phase two. Values for alpha were increased in 8 of 1 1 
instances across the dimension of satisfaction, and in all instances across the dimension of importance. 

Independent groups t-tests were performed using pooled variance estimates on nine data sets where 
child-client ranking of satisfaction levels significantly differed from other groups in phase one. There 
were significant differences in four of nine data sets. 

Independent groups t-tests were also performed on the eight data sets where significant differences were 
discovered in phase one between child clients and all others in their ranking of importance. There were 
significant differences in two of eight data sets. 

Similarly, independent groups t-tests were performed on respondent groups closest to services, and 
groups one step removed from services in ranking of importance and satisfaction where significant 
differences were present in phase one data. There were no significant differences found in any of the 
seven areas when ranked for satisfaction. When ranked for importance, a difference was found in only 
one of five areas examined. 

When semi-structured follow-up interviews were conducted among 1 0 survey respondents, a high degree 
of satisfaction was reported with the content, clarity and significance of the questionnaire. When 
specifically asked about the importance of the life domain areas chosen for study, they were universally 
endorsed. The follow-up interviews yielded no significant negative comments. 

Return to Top 



Summary & Conclusions 

An increase in values for Cronbach's alpha in 20 of 22 life domain data sets represents compelling 
evidence in support of enhanced instrument reliability for the altered questionnaire that was field tested 
in Ohio. However, the performance of independent groups t-tests in phase two, failed to convincingly 
replicate the phase one study. Little new evidence in support of construct validity was yielded in phase 
two. The follow-up interview suggests a high degree of overall respondent satisfaction with the 
instrument, and the one specific question regarding the importance of included life domain areas 
provides an indicator of content validity. 

It is expected that enhanced methodology in social validation research will increase its viability and help 
to more clearly place this approach in the larger context of health and human service policy 
development. Specifically, it is anticipated that the application of psychometric measures to survey data 
will help to increase consumer participation in the design of child and family programs in Ohio and 
beyond. 

The see is dedicated to an outcome manage-ment approach that includes consumer choice and 
contributes to the overall responsiveness of services. Working closely with families, service providers 
and community groups, the Survey is being used as part of a backdrop for the development of relevant 
outcome measures across systems and life domain areas. 
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Introduction and Purpose 

The purpose of this project was to design effective and efficient outcome measures for a youth partial 
hospitalization (YPH) program. The design was required to utilize information obtained through the 
existing intake structure, as well as to provide immediate information that could assist in treatment 
decisions, treatment plaiming and the design of interventions. The resulting design must also obtain 
information across several sources and serve as a means for monitoring clients' progress. Additionally, 
the instrument to be selected required appropriate standardization and must be acceptable for use in the 
YPH setting. Finally, it needed to be cost effective and easily incorporated into the established role of 
the therapist. Once the design was developed, a pilot was initiated. This summary presents the design of 
the outcome measurement process and results of the first six months of data collection. 
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Method and Procedures 



The design described in this summary was developed and implemented in a partial hospitalization 
program setting which serves youth who experience significant emotional and behavioral difficulties. 
The YPH setting is part of a larger community mental health system of care in Dayton, OH. Individuals 
who are referred to this program demonstrate severe symptoms of emotional or behavioral disorders 
which interfere with successful functioning within the structure of the school, home and/or the 
community, however they do not require hospitalization. Treatment goals include preparing the youth 
for return to commimity settings with behaviors that are manageable and appropriate to the situation. 
Therapy includes work in groups, in individual sessions, and family sessions. Academic instruction is 
provided daily through a tutoring model. 



The Achenbach Child Behavior Checklist (CBCL) was selected to obtain behavioral data due to its 
multimodal and multi-informant format. The instrument met design criteria, because it can be 
administered to parents, teachers and youth, yields a cross-informant profile for response comparison, 
and has been standardized (Achenbach, 1991). The results provide a profile that delineates internalized 
behavior (e.g., with-dravm, somatic complaints and anxietydepression) from externalized behavior (e.g., 
delinquency and aggression). Social problems, thought problems and attention problems can also be 
identified. The scales are easy to complete, and results can be compiled quickly with the aid of computer 
scoring. It was felt that it would be feasible to incorporate information provided by the CBCL into 
treatment planning and intervention design. 



Client information from the initial intake 

and existing client records contributed to the comprehensiveness of the outcome measures. The 
following variables were considered: gender, ethiucity, age at admission, initial primary diagnosis, 
initial secondary diagnosis, most frequent medications used, length of stay, global assessment of 
functioning (GAF), and use of additional services. 
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Clinically, it was expected that three specific areas would be included in the development of acceptable 
outcome measures: level of functioning, severity of symptoms, and client satisfaction. Table 1 describes 
the resulting evaluation design. 

Two data collection points were established. The first occurred at admission and the second during a six 
month review. The data were analyzed using a paired samples T-test to determine whether significant 
change occurred during the first six month of treatment. Results from the CBCL were compared to 
determine specific areas of significant change. 
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Results 

The results of the paired samples T-Test for the initial and six month data provided insight into the 
effectiveness of the YPH intervention. The parent or primary caretaker reported a significant reduction 
in the youths' anxious/depressed behavior, aggressive behavior and externalizing behavior (see Table 2). 
The teachers working with the child in the treatment milieu reported a decrease in externalizing (acting 
out) behavior. The most significant finding related to the youth's perception of their own behavior. 

Youth reported that after six months they experienced less withdrawal, a decrease in social problems and 
a decrease in attending problems. The youth self report total scale indicated significant changes across 
these measures. 

The initial placement profile that resulted from the collection of demographic data is shown in Table 3. 
The initial profile of the client was considered in concert with the CBCL measures. The CBCL's 
Internalizing and Externalizing scales helped to identify specific behaviors related to the diagnosis, 
allowing the treatment plan and intervention selection to be directly related to the symptomatology. 
Initially, it was anticipated that Attention Deficit Hyperactive Disorder (ADHD) would be the most 
common primary diagnosis. The results of the pilot study, however, identified Dysthymia as the most 
frequent primary diagnosis, and Oppositional Defiant Disorder as the most frequent secondary diagnosis. 
It appeared that therapeutic intervention for depression might be as important as redirecting the 
externalizing behavior. Considering the most frequent primary and secondary diagnoses, the most 
frequent medications used were as expected&shyp;Prozac (anti-depressant) and Ritalin, often prescribed 
to address externalizing behavior. These results suggested that a combination of interventions was 
necessary to promote positive outcomes, including psychotherapy, behavior change and medsomatic 
services. Results also supported the utility of a transdisciplinary approach to service provision. 

The 180 day length of stay (LOS) was identified as the baseline measure for program planning. To 
impact LOS, increased services to support the successful transition of clients to the least restrictive 
appropriate environment must be developed. Case management and foster care services are utilized in 
conjunction with YPH program within the system of care model, and can help support these transitions. 
Collaboration with the schools will also be a critical link to accomplish this goal. 

Additionally, the results indicated that the lower the score on the GAP upon intake, the longer the length 
of stay. This information will help predict the level of financial support required from funding sources 
for individual clients. In future studies, it is anticipated that examination of these factors will yield an 
anticipated LOS upon intake. 
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Discussion 

It is likely that comprehensive services for youth who have severe emotionally disturbances support the 
ultimate goal of preventing future inpatient hospitalization and residential care. 

The YPH program will use results of the outcome measures to focus their efforts in making the 
continuum of care accessible to children and adolescents in need of mental health services. Program 
standards will be reviewed to ensure that age appropriate and/or developmentally appropriate types of 
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services are made available. Skill groups, for example, could meet the needs of a specific age group 
(e.g., sex education, anger management, or independent living skills). 

The initial results of this study have had a great impact on refining the direction of the YPH program. 
The information from this study will be used in both treatment planning and program development, and 
should gamer agency support of the outcome measure process. Ultimately, information obtained from 
outcome measures will be critical in the justification of managed care dollars and authorization of 
services for children, adolescents and their families. 
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Introduction/ Purpose 

As systems for care for youth with serious emotional disturbances are designed and implemented, 
deficiencies of the existing methods of service planning have been identified. Some methods use 
language that is unfriendly to families&shyp;medically oriented categorical diagnoses, for example, 
create barriers for families, and may not guide appropriate professional intervention. Some are 
operationally explicit, but do not accommodate or embrace a wide range of clinical approaches. 

Some methods of service planning do not take into account the purposes of different systems (e.g., child 
welfare vs. special education). Because of these shortcomings, systems of care, despite efforts to achieve 
structural integration, often find it is difficult to achieve functional integration&shyp;that is, operational 
approaches to service planning that can be used throughout and across a system of care. 

This summary presents evaluations of a method of service planning that responds to these problems. 

This method was designed to be operationally explicit, family-friendly, conducive to the integration of 
clinical experience and multi-agency competency into service planning, and use of a language of service 
planning that will "work" across systems. 

Focal, factor-based service planning (FFSP) was developed in a hospital setting and has been used in a 
community-based agency (Harper, 1989; Harper & Cotton, 1991). FFSP assumes that service planning 
can be explicit and operational, comprehensive and coherent. It's design is based on the assumption that 
it is possible and preferable for parents and youth to participate in the development of the service plan. 

In addition to family participation in planning, FFSP values the use of least restrictive alternatives, and 
parsimony&shyp;the definition of modest goals at each stage&shyp;doing what needs to be done, not all 
that could be done. 

FFSP begins with selection, from among many potential problems, of a single Focal Problem, which is 
explicit, actual, and defined in terms that all parties can agree on. A corresponding Goal is then defined. 
Contributing Factors, those elements in the life of child, family, community or systems of care felt to 
contribute to the Focal Problem, are identified, and Intervenable Factors chosen. Corresponding 
Interventions, Objectives and Target Dates can then be defined. The Focal Problem and Goal and the 
Selected Factors must be stated in language that can be shared with the youth and with parents. The 
overall plan is guided by an explicit Discharge Picture. Criteria for Focal Problem, Goal, and Factors are 
given in Table 1; questions to use to check on the plan are given in Table 2. 

The Focal Service Plan does not include everything knovra about the child; it complements and does not 
take the place of clinical diagnoses; and is not a complete problem list. 

This summary presents results of the evaluation of FFSP in hospital and community settings and 
suggestions for implementing its use in systems of care. 

Return to Ton 
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Method/ Procedures 

For this evaluation research, two sites were chosen. One, an inpatient psychiatric service in a teaching 
children's hospital, had used FFSP (there, called Focal Inpatient Treatment Plaiming, FITP) for several 
years (Harper, 1989; Harper & Cotton, 1991). The other, a community-based family service agency, had 
begun using FFSP in the previous year. The staff and trainees in these two sites represented a wide range 
of experience and came from multiple disciplines. 

The evaluation questions were, (1) What perceived advantages does FFSP have for clinicians? (2) What 
is the impact on clients? Is FFSP relevant to long-term outcome? (3) How do experienced and trainee 
clinicians learn to use FFSP? and (4) How is the method used in multidisciplinary meetings? Data were 
gathered through observations of team meetings and semi-structured interviews with clinicians by one of 
the authors (J. Barnes). 
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Results and Implications 

Observations in team meetings and comments in interviews indicated that FFSP was indeed being used, 
both in home-based and inpatient services. It was experienced as useful in both settings, for example, in 
identifying the imique aspects of each case, and decreasing the risk of "losing the forest for the trees" in 
complicated cases. Respondents reported that they appreciated having a language that could be shared 
among family members and providers from different disciplines. 

FFSP was reported to have empowered clients by stating the work to be done 

in terms that youths and parents could understand and by emphasizing strengths and reachable goals. It 
stated mental health problems in non-pejorative language that could be shared with other family 
members. It was felt to facilitate work between clients and clinicians, and to clarify targets for other 
professionals. 

Moving beyond these preliminary, subjective assessments, issues for further research included 
determining optimum training strategies; finding the best methods for generalizing use of FFSP between 
settings; assessing clinician satisfaction and advantages for clients in a more systematic way; and 
evaluating the contribution of FFSP to longer-term implementation of goals. Semi- structured interviews 
with clinicians differing in experience, discipline, and setting, interviews with children and parents, and 
observations in teams will help to address these issues. 

In the meantime, the use of FFSP will respond to external incentives. There is commimity pressure for 
participation in plarming, while accrediting and reimbursing agencies increasingly require evidence of 
multi-disciplinary treatment plarming with explicit goals and objectives. The use of the FFSP 
methodology should also increase in response to internal incentives&shyp;such as care providers' 
fhistration at getting "bogged down" in data-rich cases where a goal is difficult to define, and their desire 
to pool input from professionals and commimity members alike. 

As to implementation, our observations suggest several necessary conditions. On the institutional level, 
these include a mandate for explicit, participatory service planning; and the provision of needed 
resources, in terms of time, training, hardware, and the development of forms suited to this kind of 
service plarming. Among persoimel, implementing FFSP requires opeimess to learning new ways to 
organize clinical data, perhaps divergent from traditional practice; the capacity at each phase of the work 
to step back and reflect on the plarming process itself; and a commitment to consensus development and 
participatory service plarming. 

Implementation requires that staff development be thought of as a process, not as a one-time training 
session; that it emphasize learner-directed, experiential learning; and de-emphasizes paperwork, or 
"getting the right form" as the solution to the challenges of service plarming. Training must address 
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critical knowledge, skills, and attitudes. Staff must learn that categorical diagnoses do not determine the 
content of treatment; they must learn how to use criteria for defining the problem being treated, the 
contributing, and the intervenable factors; and they must learn that obfuscation in service planning is not 
only a problem in documentation, but in clinical practice as well. They must become skilled in 
monitoring the planning process, listening at each phase for the degree of consensus, and in defining 
factors, especially intervenable factors, out of clinical data. They must adopt the values of critical 
listening, parsimony in planning, and participatory planning. 

Return to Top 



References 

Harper, G., (1989). Focal Inpatient Treatment Planning. Journal of the American Academy of Child and 
Adolescent Psychiatry 28,31-37 

Harper, G. P. & Cotton, N. S. (1991). Child and adolescent treatment. In L. I. Sederer (Ed.), Inpatient 
Psychiatry, 3rd ed. Baltimore: Williams and Wilkins. 

Return to Top 



Gordon Harper, MD 
Director, Inpatient Psychiatry 
Department of Psychiatry, 

Children's Hospital 
300 Longwood Avenue 
Boston MA 02115 
617/355-7638 Fax: 617/734-3751 
EMail: harperg@al.tch.harvard.edu 

Christina Crowe, M.S.W. 

Director, Community Based Programs 
Judge Baker Children's Center 
295 Longwood Avenue 
Boston MA 02115 
617/232-8390 ext. 2609 
Fax: 617/232-4125 

Jacqueline Barnes, Ph.D. 

Associate Director 
Technical Assistance Center 
Judge Baker Children's Center 
295 Longwood Avenue 
Boston MA 02115 
617/232-8390; FAX 617/232-4125 

*Dr. Barnes has changed employment. Correspondence should now be addressed to Jacqueline Barnes, 
Ph.D., Senior Lecturer, Department of Child and Family Mental Health, The Royal Free Hospital School 
of Medicine and the Tavistock Clinic, London University, Royal Free Hospital, Leopold Muller 
University Department, Rowland Hill Street, London NW3 2PF England, Tel 0171 794 0500 ext 5718 
Return to "Pop 



O 

ERIC 



423 



12/11/98 1:42 PM 



http://lumpy.tmhi.usf.edu/CFSroot/rtc/proceeding8thy8th.379tlJ 






xt 



Ln 

cv 







m 









o 

w> 

iiaj 







o 



12/11/98 1:21 PM 



§ 

XI 

d 

ON 

r- 

cn 

X 

•4-* 

00 

:a 

4-* 

00 

Oi) 

.s 

T 3 

< 1 > 

< 1 > 

o 

o 



o 

t: 

o 

o 

Vh 

CO 

CJh 

u 

T 3 

< 1 > 



X 

•-g 

cx 



B- 

X 




o 



00 

ON 



<N 



c\? 




CO 

c\? 




00 



8th.383.html 



http://lumpy.fmhi.usf.edu/CFSroot/rtc/proceeding8th/8th.383.htmI 



I i-iome 1 1 searcn 1 1 index I 

8th Annual Research & Training Center Conference Proceedings, Dept of Child and Family Studies, 
Florida Mental Health Institute, University of South Florida, 1996 

An Alternative Approach to the Study of the Children’s Mental 
Health Service Systems: Anthropological Models 

Authors 



Introduction Method Discussion References 
Back to Table of Contents 



Introduction and Definition 

This summary explores the use of medical anthropology models to explain the relationships between 
parents of children labeled seriously emotionally disturbed and service providers. Today the values and 
practices of community service providers are moving away from institutionalization and toward 
community care and "individualized service models" (Bums & Friedman, 1990:95). This shift includes 
the recognition that the family of a child with a serious emotional disturbance requires as much support 
as does the child. Of equal importance, parents are becoming recognized and valued as the experts in 
their children's problems and needs, and are beginning to participate in service planning for their 
children. 

In the new individualized service model, "the concept of a partnership between parents and professionals 
expands the more traditional roles of 'patient' and 'client' the parents tend to assume" (Friesen & 

Koroloff, 1990). As this service model becomes more accepted by all services and providers in the 
children's mental health service systems, a better understanding must be gained about what is happening 
in the relationships between the parents and providers. 

Medical anthropology has a number of established theoretical models that address the relationship 
between patients and healers, which are used here to examine this relationship in the children's mental 
health service systems. Medical anthropology models are sociocultural models that recognize that the 
system's context is necessary in studying the strategies a family uses to seek and obtain care and support 
for a family member with medical or psychological health problems. 
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Method 



Medical anthropology models were used to analyze families' strategies for seeking and interacting with 
the available services, such as the special education schools explored in the case study. Data analyzed 
were from a qualitative multiple-case study of two families. In depth interviews, conducted in 1992, with 
young men and mothers explored their interactions with the schools and other service providers. The 
families were participants in the National Adolescent and Child Treatment Study (NACTS), a research 
project conducted by the Research and Training Center for Children's Mental Health, Department of 
Child and Family Studies, Florida Mental Health Institute (FMHI), University of South Florida (USF). 



Anthropology of Biomedicine 

The Anthropology of Biomedicine model is primarily cognitive. This approach and model "focus on 
what sickness and health mean to individuals and how, as a consequence of these meanings people act in 
specific ways regarding treatment" (Lazarus, 1988). Kleinman (1980), had developed an explanation for 
the anthropology of biomedicine model. He uses the concept of "explanatory models", which are notions 
held by the patients, families and practitioners about specific episodes of illness (Kleinman, 1988). 
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Explanatory models may include cognitive, affective, moral and social concerns (e.g., specific health 
beliefs, fears and concerns about health, stigma of SED label, etc.) 

For the child with a serious emotional disturbance, the lack of mutual imderstanding on the part of 
his/her family and the school personnel can cause miscommunication resulting in long-term 
consequences for treatment decisions and outcomes. The Anthropology of Biomedicine model is well 
suited to explain miscommimications and family noncompliance with the school personnel's treatment 
recommendations. 

EXAMPLE - Case Study Ah Mother, Dana Mae Dodd; Son, Chuck Dodd; and Special Education 
Teacher, Ms. M. 

Dana Mae Dodd and Ms. M. struggled to imderstand and work with each other. They each had their own 
perspective and explanatory model. 

Dana Mae expected to have a lot of say and influence in how Chuck was treated at the day treatment 
facility. She kept in close contact with the teachers and expected to hear about problems on a daily basis, 
if needed. Dana Mae was assertive, she said "... I got what I wanted because I demanded it. I didn't ask, I 
demanded it when it came to my young-im...over the years the teachers and I have learned to work 
together." 

Ms. M. explained how irritating Dana Mae's behavior could be when she called constantly and sent notes 
to school that were not written clearly. Ms. M. acknowledged that Dana Mae had her good points, 
saying, "She seems genuinely concerned. He [Chuck] does get his medicine when things get serious 
enough. ... She is better than most of the parents. I see some of the same behaviors in Chuck as I do in 
Dana Mae." 

Dana Mae's belief that she should have daily input into Chuck's treatment often clashed with Ms. M.'s 
beliefs on how best to teach Chuck. This led to tension between the two women and problems in 
providing Chuck the best care. 

Critical Medical Anthropology 

The Critical Medical Anthropology model incorporates "political and economic determinants of society 
into studies of health and medicine" (Lazarus, 1988:45). The personal experience is seen as taking place 
in the social context of the society as a whole . Health is defined in this model "as access to and control 
over the basic material (e.g., transportation, medication) and non-material resources (e.g., education, 
health information, therapy) that sustain and promote life at a high level of satisfaction" (Baer, Singer, & 
Johnsen 1986). 

The critical medical anthropology model places the family-service provider interactions in the context of 
the children mental health service systems. The context influences the available services, beliefs about 
mental illness in children, and the "asymmetrical social relations based on social class and authority", 
which all become critical parts of the analysis and model (Lazarus, 1988). 

EXAMPLE - Case Study B: Mother, Beth Brennan; and Son, Calvin Brennan 
Beth wanted to know what was wrong with Calvin. At one time she had wanted him to have a 
neurological examination. She said "I wanted the examination of the brain, the head, to see if, where the 
problem was coming from. If it was something mental, psychological, something I did, something I did 
when I was pregnant ... or anything. I wanted more help. Medical help or advice which we didn't get." 

She explained why they did not get more help: "Money I guess. ... Nobody ever recommended it. It 
wasn't paid for through the school and we didn't have the money to do it ourselves. His insurance didn't 
cover stuff like that." The asymmetrical social relationship between the school personnel and Beth meant 
that Beth felt she did not have the right or knowledge to question the actions of the teachers and 
administrators. Calvin never did get a neurological exam because of economic, communication, and 
service system barriers. 
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Summary and Implications 

In the children's mental health service system there is grotving recognition that there must be a true 
partnership between the parents and service providers. There is also a conceptual shift towards families 
of the mentally ill in terms of "a model of stress, coping, and adaptation [that] views familial behaviors 
as coping strategies," rather than as interfering wdth prescribed treatments, or as toxic agents (Lefley, 
1989). 

Medical anthropology models are well suited to examine what happens within the relationships 
developed between the families of children with mental health problems and the services providers, e.g. 
special education schools. Of particular interest was using the models to examine the family and school 
personnel interactions and their ability to cooperate in the care of the child. The models are also useful in 
exploring the reasons why the family and child did or did not comply with the treatments and 
instructions provided by the schools. 
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Information and Referral 

Information and Referral (I «&; R) is a system designed to link a person in need of information about 
human services with needed resources. "I &. R represents a continuum of functional elements ranging 
from information assistance and advice-giving to referral and follow-up" (Levinson, 1987, p. 914). I «&; R 
services generally provide information to individuals who telephone an agency, describe a problem, and 
request assistance. Trained staff assess the nature of the problem(s), provide information, and advice, 
appropriate referrals to existing resources, and follow-up to determine if services were obtained. 

"Varying levels of counseling and advocacy are offered in accordance with the user's needs and the 
mission of the I «&; R organization" (Levinson, 1987, 

p. 914). There are several different types of I R services but most can be classified as one of the 
following: 

General Assistance: agencies that serve all segments of the population and deal with a wide range of 
social problems. 

Specialized: agencies designed to meet the needs of a specific target population (e.g., individuals with 
disabilities), to deal with a significant social problem (e.g., HIV/AIDS), or to provide a specific service 
(e.g., crisis intervention). 

Full Service: agencies that serve all segments of the community and address a wide range of social 
problems; in addition, these agencies conduct and participate in a variety of community planning and 
needs assessment activities to improve the continuity of care in the human service system. 

The philosophy of I «&; R is that being "information poor" is a handicap in obtaining services; being 
"information rich" enables one to reap the benefits of a service system (Levinson, 1987, p. 919). One 
goal of I R agencies is to provide information equity so that individuals in need of help can obtain 
services in a timely manner. 
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Importance of I & R Services 



Early intervention is considered to be an important factor in the successful treatment of at-risk children 
and their families. Unfortunately, it is often difficult for people to identify and locate services in the 
early stages of a crisis. For example, 57% of respondents in a community needs assessment study 
reported that they did not get help for a personal or family problem because they "thought no help was 
available" (ICOPE, 1992). Whereas only 10% of the general citizenry in another study reported that they 
did not know where to find the help they needed (Balducci, 1994), this rate was 16% for low-income 
respondents, 23% for individuals who had experienced a serious problem within the last year, and 45% 
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for Spanish- monolingual respondents (Montenegro &, Batsche, 1995). Lack of familiarity with service 
systems and service providers is a major barrier that reduces opportunities for early intervention. 

In a recent article on early intervention with children with serious behavior problems, Dunlap and Fox 
(1995) describe "family support" as one of the major elements of comprehensive early intervention 
services. They refer to family support as "any and all actions that serve to strengthen the family system, 
especially as these actions pertain to the family's assimilation and understanding of their child's 
disability" (Dunlap &, Fox, 1995, p. 18). Among the supports they describe are the need for information, 
education/training, and social and emotional support. Although the provision of on-going intervention is 
beyond the scope of I «& R services, these organizations do provide information to help with the initial 
identification of service providers, support groups, and community activities. I «& R counselors are also 
trained to teach families how to operate within a community-based social service system, i.e., 
self-advocacy training. Finally, I «& R counselors serve as a source for help for individuals in need of 
emotional support for long-term problems and stresses. 

In summary, I «& R services are often the entry point into the system of care for children and families. 
These agencies can provide individualized ongoing support and services ranging from self-advocacy 
training to 24-hour social and emotional support. Little has been published describing the role of I & R 
agencies as part of early intervention strategies, a family support service, or a critical component of the 
continuum of care. The objectives of this project were therefore to: 

1. identify sources of information used by individuals when they are in need of help; 

2. identify barriers to obtaining information about community services that may prevent individuals 
from seeking early intervention assistance; and to 

3. conduct an analysis of "counseling" calls received by an I «& R agency to determine the nature of 
the crisis, the extent of intervention provided by phone counselors, and the extent to which 
counseling calls were referred to other service providers. 
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Procedures 



The methods used to address objectives one and two were focus groups, key informant interviews, and a 
survey of citizens. Focus groups and interviews were conducted with consumers of mental health and 
other social services and representatives from community social service agencies and organizations, 
state/county social service agencies, information-service agencies, and community churches and 
synagogues in Pinellas County, Florida. In selecting consumers to participate in the focus groups, target 
populations were identified who were representative of the demographic and cultural composition of the 
community including: Southeast Asian consumers, African-American consumers, and the elderly 
population. The social service providers included representatives from community schools, hospitals, 
public health centers, child care centers, domestic violence shelters, family service centers, mental health 
centers, and substance abuse treatment centers. The state/county agencies included representatives from 
Health and Rehabilitative Services, Juvenile Justice, Public Health, and Social Services. The participants 
in the focus groups with social services providers and state/county agencies were selected in consultation 
with members of the Pinellas County Interagency Council on Planning and Evaluation. Agencies were 
invited to participate who had extensive experience in dealing with individuals seeking assistance from 
public sector programs and services. 

Finally, a survey of the general citizenry was conducted to identify the information-seeking behaviors 
and preferences of citizens of Pinellas County. One hundred citizens, selected at random at a community 
shopping center, were asked to identify the information sources they would use if they were in need of 
help. 



The analysis of the counseling calls (objective three) was conducted by reviewing the case records of all 
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counseling calls received by a community I & R agency in the month of June, 1994. The agency defined 
counseling calls as those in which emotional and social support was provided to individuals 
experiencing distress yet the problems were not so severe as to be classified as a crisis intervention call. 
Of the 4,200 calls received by the agency in the month of June, 1994, 520 were classified as counseling 
calls. The case records of these 520 calls were analyzed for gender and age distribution, nature of calls, 
length of calls, and referral rates of counselors. 
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Results and Implications 



Sources of Information and Barriers to Obtaining Information 

The first two objectives of the project were to identify sources of information used by individuals when 
they are in need of help and to identify barriers to obtaining information about community services. The 
major findings included the following: 



• The majority of individuals in need of help first seek information from family members and 
friends. I & R agencies were cited as the second most frequent source of information followed by 
the yellow pages, physicians, police, newspapers, clergy, counselors/caseworkers, and service 
providers. 

• Approximately 10% of the respondents in the citizen survey (N = 100) were unable to identify any 
source of help; 23% were unable to identify more than one source of information. 

• Age differences were found in the identification of the first source of help. For all youth, 100% of 
the respondents indicated they would seek information from a friend as a first source of help and 
92% stated they would also seek information from a family member. Teenagers were less likely 
than all other age groups to seek information from a member of the clergy or a physician. 

• Barriers to trying to obtain help from the human service system were issues of pride and shame, 
lack of trust, and concern for privacy. 

• System barriers cited were fragmentation of services, transportation problems for those needing 
help, application processing delays, categorical funding, bureaucratic obstacles, and lack of 
available, accessible services. 



• The predominant cultural barriers among Hispanic/Latino citizens was the lack of services for 
individuals with limited-English speaking ability and the fear of deportation. 

• African-American respondents cited barriers due to lack of service providers who were 

• African-American or who were responsive to the African-American community. 

African-American respondents said they were more likely to turn to family members, neighbors, 
and clergy in times of trouble or crisis rather than to service agencies. 

• Lack of bilingual ability and illiteracy were reported to be the major barriers to knowing about and 
using human services for the Southeast Asian community included in this project. Suggestions for 
providing information to these individuals included working with religious leaders (Buddhist 
monks, the refugee resettlement program sponsored by Catholic Charities, and various programs 
sponsored by the Protestant chirrches in the community), advertising on the local Southeast Asian 
radio station, and disseminating materials in appropriate languages at the Asian markets in the 
community. 




Barriers identified by service providers and information providers were the lack of common data 
bases containing current and accurate information for use by all information providers, the 
proliferation of I & R agencies, and the duplication of services by I & R agencies with similar 
purposes. 
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Counseling Call Analysis 

The third objective of this project was to analyze the "counseling" calls received by an I & R agency to 
determine the nature of the problem, the extent of intervention provided by phone counselors, and the 
extent to which counseling calls were referred to other service providers. The major findings of this 
analysis included the following: 

• A higher percentage of females (66%) than males (34%) sought support from the telephone 
counselors during the period of the analysis. 

• The callers ranged in age from 17 to 67. In 88% of the records, the caller was the client; in 12%, 
the individual was calling on behalf of a child (6%) or another adult (6%). 

• The highest percentage of calls fsee Figure If were for emotional problems (30%), problems with 
a child or teenager (15%), substance abuse problems (11%), partner abuse (9%), financial 
problems (8%), child abuse (6%), suicidal thoughts (6%), relationship problems (4%), sexual 
assault victim (3%), health problem (2%), homelessness (2%), and work problems (2%). 

• The shortest counseling call (1 minute) was made by a client requesting counseling services for 
sexual abuse. He did not want to talk to the I & R personnel, although he was quite distressed. He 
was referred to a local Abuse Center. 

• The longest counseling call (45 minutes) was made by a client whose fiancee had been jailed for 
drug possession. She was shocked, angry, and distressed over the situation but wanted to preserve 
the relationship. She was referred to Nar-Anon. 

• The telephone counselors offered referrals to approximately 50% of the callers. When referrals 
were not made, it was because no service was available or because a referral was not wanted or 
not appropriate. 
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Implications 

The outcomes of this project documented that I & R agencies serve as an important entry point that can 
facilitate early intervention services. However, the most frequently cited information sources were 
family and friends. Efforts should therefore be made to ensure the general citizen^ is informed of 
community services and I & R agency services so that individuals and families will be able to obtain 
help when needed. It is imperative that I & R agencies have current and accurate information about 
service providers and community organizations and that they be trained to disseminate this information 
appropriately. There is also a need for communication linkages among the various information providers 
to avoid duplication of effort and to reduce caller frustration in seeking assistance. 

The lack of bilingual and bicultural staff is a major obstacle to seeking help. There is a need to provide a 
wider variety of I & R services in culturally responsive ways to meet the needs of the different ethnic 
populations of local communities and to conduct outreach activities to help individuals in need of 
service locate and obtain assistance. 

I & R agencies perform an important role in providing social and emotional support to individuals in 
distress as well as in times of crisis. It is therefore critical that these agencies be considered to be an 
important component in the system of care continuum and that they be included as one of several 
participants in the provision of family support. I & R agencies can be an important link in achieving the 
goal to provide information equity among at-risk children and their families. 
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Introduction 

One of the primary aims of the Vanderbilt School-Based Counseling Evaluation Project is to assess 
whether school-based mental health services increase the accessibility and use of services among 
children with serious emotional disturbance relative to traditional community-based services. This 
project presented a poster at the 7th Annual Research Conference for Children's Mental Health in which 
the service utilization rates of the project's school-based and community-based counseling groups were 
reported. At that time, the utilization rates for these two groups were highly discrepant in favor of the 
school -based counseling group. While the participation rate for school-based counseling was 99% (9 1 
families), that for community -based counseling was only 17% (17 families). 

In an effort to increase participation among families referred for community-based services, the project 
has implemented several significant changes in the recruitment and referral process. Instead of paying 
the customary fee for service to the community mental health center, the project now pays a portion of 
each therapists' salary to cover those hours devoted to scheduling and providing counseling services to 
participating project families. As a result, the project has been successful in recruiting a total of 46 
(38%) families for community-based services. The finding that the rate of service acquisition for the 
community-based counseling group continues to be significantly lower than that for the school-based 
counseling group (i.e., 85%) strongly suggests that the school-based program provides mental health 
services to children who would otherwise go without needed services. 
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Methods & Results 

In order to understand better why some families seek community-based services and others do not, t-test 
comparisons were made on a number of theoretically determined variables. 

No differences were found, however, on any of the following demographic variables: child's age, gender, 
or race; family income; education of primary caregiver; presence of a male in the household; prior use of 
mental health services; or the extent to which the family reported having reliable transportation. 
Likewise, no statistically significant differences between groups were found on a measure of parental 
distress (Brief Symptom Inventory; Derogatis & Melisaratos, 1983), a measure of parent involvement in 
the child's daily life, or a measure of the frequency with which various strategies are used to resolve 
conflicts between the parent and identified child. 

Since primary caregivers are instrumental in obtaining services once a referral is made, their perceptions 
of child functioning were expected to distinguish between those who follow through on the referral for 
services and those who do not. A comparison of scores on the parent version of the Child Behavior 
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Checklist (CBCL; Achenbach, 1991) showed a trend indicating that seeking treatment was associated 
with higher (i.e., more disturbed) CBCL scores (t = -1.83, p = .07). This was also the case for 
externalizing symptoms (t = -1 .67, p = .09), but this finding did not hold true for internalizing symptoms 
(t = -0.50, p = .61). Likewise, parent perceptions of their child's interpersonal strengths also 
distinguished between the two groups. Parents who sought services for their child reported significantly 
fewer interpersonal strengths for their child than those parents who did not obtain these services (t = 

2.20, p < .05). 

In addition, scores on the Family Assessment Device (FAD; Epstein, Baldwin & Bishop, 1983) were 
significantly different for service utilizers and nonutilizers. Families who sought treatment had 
significantly lower mean scores on the General Functioning Index of the FAD than did those who had 
not yet obtained services (t = 3.24, p < .001). This was also the case for the Problem-Solving subscale (t 
= 3.59,p < .001), the Affective Responsiveness subscale (t = 4.41 .P< .001), and the Behavioral Control 
subscale (t = 3.02, p < .01) of the FAD. Other subscales of the FAD that were not significantly different 
between the two groups included Affective Involvement, Communication, and Roles/Responsibilities. 

In an attempt to learn why some families decided against accessing the services for which they had been 
referred, a Nonattendance Questionnaire was administered during the project's most recent home 
interview (i.e.. Summer, 1994). In light of the finding that families reporting higher family functioning 
were less likely to access services, it is interesting to note that among the most frequently endorsed items 
were "I can handle my child's problems on my own," and "Children's problems tend to work out by 
themselves." Likewise, the high frequency with which the item "My child didn't really have a problem" 
is consistent with the finding that families who 

did not access services reported fewer problem behaviors for their children on the CBCL. 
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Introduction 

Previous studies in Psychiatry have attempted to understand the influences of city life on 
psychopathology and development in children. For the most part, these studies have dichotomized 
environments into urban and rural areas with the underlying hypothesis that all urban areas are 
experienced by inhabitants as stressful and deleterious to their mental health. However, environmental 
stress is not distributed uniformly within a city&shyp;there are safer areas and less affluent areas. 
Therefore, wholesale comparisons of urban versus rural populations may not make sense. Mapping as a 
methodology offers an alternative wherein the micro-environments that neighborhoods offer can be 
examined with more precision. These environments can then be compared with the populations that live 
within these neighborhoods. 

With these methodological considerations in mind, one can begin to examine how neighborhood 
environments are related to children's mental health and the use of mental health services. The associated 
hypothesis that can be examined is one that was most explicitly stated in Unclaimed Children (Knitzer, 
1984): Distribution of mental health services is uneven and may be most scarce in areas of highest need. 
This preliminary study attempted to use mapping and statistical techniques to begin to address these 
issues. 
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Methodology 

The setting for this preliminary study was a small city of 124,000 people located in the Northeastern 
United States. The median family income in 1990 was $15,000 and 31% of families were living below 
the poverty level. Three main concepts are defined for study: (1) mental health service use, (2) mental 
health service need, and (3) the ratio of mental health service use to mental health service need. An 
estimate of mental health service use was obtained by enumerating cases seen at a child guidance clinic. 
These index cases were geo-coded and plotted within census tract block groups of the city. Child 
protective referrals were enumerated and used as an indicator of disturbances in mental health with 
respect to children and families in each census tract. Federal census tract data were used to characterize 
attributes of each census tract block group. 

In order to represent the relationship of mental health service need to service use, an index was 
constructed&shyp;number of cases enrolled in the mental health clinic per block group was divided by 
the number of child protective referrals per block group; this was done for each block group. This index 
was referred to as the mental health service use to need ratio. This index will be larger if the ratio of 
service use to service need is high. However, if there is greater need or less mental health service use, 
then this index will be lower. 
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Data for this study were gathered from one of the publicly funded child guidance clinics whose staff sees 
about 1,200 children per year&shyp; about one-third of the child mental health cases served in the urban 
center. The city was small enough in area that patients could move freely between the child guidance 
agencies. Data were also obtained from the state child protective service. Information from these two 
agencies was downloaded from the agencies' computer system to a centralized PC. In addition, US 1990 
Census data and a city wide survey of vacant housing were used. Census and neighborhood survey data 
provide proxies to characterize the neighborhood environments. 

Computer assisted mapping was done using Mapinfo, locating cases within US census tract block 
groups. Using these data sources and techniques, data for each block group were compiled and 
statistically analyzed using SYSTAT. 
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Results 



This study revealed that an average of two percent of the entire population who were less than 1 8 years 
of age received mental health services over a five year period. This figure compares favorably with 
similar surveys done elsewhere in the U.S. Maps produced from these data, however, show considerable 
variation within the city in terms of distribution of these services. Considering each of the 129 block 
groups located within the city limits, anywhere from zero to nine percent of the population received 
services over a five year period. Service intense block groups are scattered throughout the city and not 
clustered around the child guidance clinics. 

In a four month period, there were from zero to fifty child protective referrals per one thousand 
population in each of the 129 city block groups, with an average of seven referrals per one thousand 
population. Again, the density of referrals per block group shows scatter and variability. 

The percentage of vacant buildings offers one view of a city neighborhood. Other characteristics of a 
city neighborhood include unemployment rate, ethnic mix of the population, proportion of households 
that move, family income, and percentage of homes below the poverty level. These characteristics may 
be considered different aspects of urban stress. It was hypothesized that these measures of urban stress 
impact the mental health of the members of the neighborhood and hence are related to mental health 
service use and need. 



In order to examine the potential relationships between these environmental factors and mental health 
service use and need, statistical regression analysis was used. These analyses show that child protective 
referrals as a proxy for mental health service need are directly proportional to the unemployment rate 
and the percent of the population that is not Caucasian. Mental health need as represented by child 
protective referrals is not statistically related to poverty levels, proportion of the population that is 
English speaking, proportion of the population graduating high school, proportion of the population 
moving into the area in the last five years, population density, or proportion of buildings that are vacant. 

The amount of mental health service provided is directly proportional to the unemployment rate. In those 
areas with highest urban stress as measured by high numbers of vacant buildings, the amount of mental 
health services is also related to the number of people living in the same area for longer than five years. 
The density of mental health cases, however, is not related to percent of the population which is of 
minority status, population density (number of people/number of buildings), percent of high school 
graduates, percent of English spewing families, or proportion of families living in poverty. 



The use to need ratio, as was mentioned above, represents the extent to which mental health problems 
are being addressed by services. This index provides a test of whether or not the most severely stressed 
areas of the city are receiving adequate mental health services, or as suggested by Knitzer (1984), 
whether the most disadvantaged children are receiving fewer services than less disadvantaged children. 
In fact, statistical analyses reveal that a high child protective referral rate is associated with a low index 
use to need ratio, providing confirming evidence for the hypothesis that those most in need are not 
receiving mental health services. A high proportion of members of ethnic/racial minorities within the 
population is also associated with a low use to need index. This result indicates that non-minority 
populations avail themselves of mental health services more than do members of minority populations. 
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If we consider those 1 5% of census tracts of the city that have higher stress as measured by percentage 
of vacant buildings, we discover that a high use to need index is associated with the highest percentage 
of vacant buildings and the highest population densities, in addition to the above findings. These 
findings indicate that people living in areas of highest population density and highest urban stress do 
avail themselves of mental health services when they need them. 

What is equally important to note is that the use to need index is not related to employment, migration, 
poverty levels, percentage of the population that speaks English, or percentage of the population who 
has graduated high school. These negative findings demonstrate that when child protective referrals are 
high, children from that neighborhood are using mental health services, whether or not they are poor, 
speak English, have graduated high school, have recently moved or are employed. 

These findings are preliminary, especially since they use data from only one of three public mental 
health clinics. Therefore, these findings need to be confirmed by additional data collection and analyses. 
Return to Top 
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Introduction/Purpose 

Developing the means to address barriers to help seeking is critical given that lower income, minority 
children are at greater risk for the development of psychopathology. This paper outlines a series of three 
research studies designed to identify factors related to children's mental health service usage and barriers 
to help seeking for urban minority children and their caregivers. In addition, this paper describes the 
systematic development and evaluation of a telephone intervention strategy aimed to increase overall 
attendance at initial intake appointments at an urban children's mental health agency. The first study 
explores differences in demographic variables, for two groups of children, those that came to an initial 
intake interview and those that requested child mental health services, but failed to come to any 
scheduled appointments (n=450). The second study evaluates a telephone engagement intervention 
meant to increase initial attendance (n=54). Finally, the third study more rigorously evaluates the impact 
of an intensive telephone intervention on initial attendance rates by randomly assigning families to the 
more focused telephone intervention or a "business as usual" telephone screening (n=108). 
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Method/Procedures 



The series of studies were meant to reach the following objectives: 1) to document demographic 
characteristics of those children and families who requested mental health services at an urban agency; 

2) to explore differences between those children and families that requested services but did not come 
for an intake appointment and those that came for an intake appointment; 

3) to identify factors that would predict attendance at the initial appointment and; 4) to develop and 
evaluate a telephone intervention strategy based upon the empirical findings of previous mental health 
services literature. 



Research Site 

All three studies were conducted at the Institute for Juvenile Research (UR), the Child Psychiatry 
Division of the University of Illinois at Chicago. UR is an inner-city, child mental health agency, with 
67.3% of the children living with their mothers in single parent households. Approximately, 85% of the 
450 families who requested services last year were supported by Public Assistance. Almost two thirds of 
children seen at the agency were African American, 

12% were Latino and the remaining portion were Caucasian. 



Study #1 

Of the 450 children who were referred to 

the Institute for Juvenile Research and an intake appointment was scheduled, the following demographic 
information was obtained: age of identified child, gender of child, primary and secondary caregivers, 
presenting problem and, court involvement. These children were then tracked through the intake system 
at the agency. 
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Study #2 

The next study aimed to systematically develop and evaluate a telephone intervention strategy with the 
primary goal of increasing overall attendance at initial intake appointments. Two master's level interns, 
under the direction of a clinical faculty member implemented the 30-minute telephone engagement 
intervention. This intervention focused on: helping the primary caregiver invest in the help-seeking 
process by clearly identifying their child's presenting difficulties; framing caregiver's actions as having 
the potential to impact the current situation; and having the caregiver take some concrete steps to address 
the situation, even prior to the initial appointment. In addition, the inter-vention was intended to 
systematically explore barriers to help seeking, both within the family and the environment. Finally, an 
active problem solving approach was used to develop the means to address obstacles to contact with the 
agency. 

A sample of twenty-seven telephone intervention cases were compared to the first twenty-seven cases 
scheduled for intake appointments in the same month of the previous year. 

Study #3 

Next, in order to address some of the limitations of the previous study related to sample size and design, 
the investigator evaluated the telephone intervention strategy by randomly assigning 108 new requests 
for child mental health services to one of two conditions. In the first condition, 55 telephone intakes 
were assigned for a thirty minute, intensive engagement intervention with two Master's level social 
workers. The second condition consisted of a routine telephone screening lasting approximately 
30 minutes conducted by a third master's level 

social worker (n=53). This screening related to the presenting problem of the child and appropriate fit for 
the agency. 
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Results 



Study #1 

Only 277 (61.6%) of the 450 children accepted for intake appointments were ever brought in by their 
caregiver. In relation to gender of the child, 64.5% (n=290) of all children accepted for service 
at the agency were male, however, only 176 (60.7%) actually came for an intake appointment. In 
comparison, girls were more likely to be brought to at least one appointment. Of the 160 girls accepted 
for intake appointments, 101 (63.1%) actually came to at least one appointment. When primary caregiver 
was considered, 302 children (67.1%) were being cared for by their mothers at the time of the initial call 
to the agency. Only 170 of these children (56.3%) were ever seen for a first interview. Children who 
were parented by foster care givers were more likely to come in for scheduled appointments, 1 1 9 intakes 
were scheduled by foster parents and 91 (76.5%) of those appointments were kept. 

In relation to presenting problem, children were most likely to be referred for aggressive behavior 
(n=106 intakes scheduled) or ADHD symptoms (n=79). However, only 62 (58.4%) of children with 
aggressive behavior and 35 (44.3%) of children presenting with ADHD features ever came for an initial 
appointment. Children identified as having been severely abused or experiencing severe trauma were 
more likely to be brought to an appointment&shyp;22 appointments scheduled vs. 16 (72.7%) 
appointments kept. Children who did not have court involvement more often contacted the agency 
(n=309), but only 173 (56.0%) of these children were ever brought to the agency. Families who were 
involved wdth the Juvenile Court were more likely to follow through with a scheduled intake 
appointment &shyp;103 (75.2%) of the 137 intake appointments were kept. 



Study #2 

Results reveal that the engagement strategy increased initial appointment attendance by 29% in 
comparison to the more traditional telephone intake procedures. Of the intervention families, only 6 did 
not come to an initial appointment, in comparison to 14 no shows for the same month the preceding 
year. Chi square analyses revealed a significant increase in intake attendance for the intervention 
children and families (c2 = 5.08, p < .05). 
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Study #3 

Of the 55 families that received the telephone intervention, 72.7% (n = 40) came to the first appointment 
or called at least a day prior to the interview to reschedule. Of those that underwent the more traditional 
screening, only 45.3% came to the appointment or called independently. Chi square analyses revealed 
significant results (c2 = 8.42, p < .01). Logistic regression was then used to explore contribution of age 
of child, court involvement, primary caregiver and gender of the child, however, these demographic 
variables did not add significant explanation. 
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Implications 

Thus far, the telephone engagement intervention discussed above has demonstrated efficacy for 
increasing initial attendance by inner-city families. However, its impact in relation to ongoing 
engagement is still being evaluated. Further research is needed to accurately estimate the cost 
effectiveness of this approach and the impact that such a shift in agency procedures has on efficiency, 
staff morale, etc. 

The foundation for any future children's mental health services research is the ability to engage children 
and their caregivers in services. 

If clients do not access services or remain in service for a sufficient period of time, the efficacy of any 
mental health treatment cannot be tested. Larger research initiatives are clearly needed if there is to be an 
increase in quality and access to care by minority children and families. In light of the growing concern 
about the inappropriate use of costly, restrictive treatment for children, this paper has begun to examine 
issues related to increased use of outpatient children's mental health services. 
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Introduction & Purpose 

The question of referral bias is particularly relevant in children's mental health, as children rarely refer 
themselves. Referral source and presenting problem are particularly germane to this question of bias as 
they reflect who perceives what kind of problem. A critical question is whether intrinsic background 
characteristics of the child, such as ethnicity, affect the process of referral and evaluation for behavioral 
and emotional problems. In the context of the burgeoning literature on the role of cultural congruency in 
mental health service delivery, it becomes important to examine the issue of bias in referring minority 
children for services. 

Building on previous studies, we were interested in examining our clinic population to explore the 
question of referral bias. Controlling for socioeconomic status, gender, and age, we wished to examine 
whether in our clinic African-American children when compared with Caucasians were: 

1. more likely to be referred by coercive referral sources (e.g., courts) 

2. have more severe (externalizing) presenting problems 

3. have similar levels of psychiatric impairment. 
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Procedures 

Subjects 

The 206 subjects were taken from the total 683 clients who contacted the Child Study Center (CSC) 
from July 1, 1992 to June 30, 1993. Forty-eight percent of the children who come to the clinic are from 
New Haven&shyp;the remaining 52% are from the outlying suburbs(34%) and the rest of the state 
(18%). Over half of the children are Caucasian (55%), 31% are African-American, 14% Hispanic. Over 
600 children are enrolled as new cases each year. 

Our sample consisted of subjects who were (a) either African-American or Caucasian, (b) were referred 
for primarily an externalizing or internalizing problem and (c) received at time of evaluation a primary 
DSM-III-R Axis I diagnosis reflecting an internalizing or externalizing disorder. Overall, these 206 
subjects represent 30% of clients who contacted the clinic during that year. The sample was 57% male (n 
= 117) and 43% female (n=89); 31% (n=64) African-Americans and 69% 

(n= 142) Caucasian. 
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Site 

The study site is a university based inner-city children's mental health clinic that accepts patients on a 
sliding fee scale according to the family's financial status. Contact begins with a phone call from the 
parent or other referring source. The intake secretary records age, gender, place of residence, school, 
family composition, presenting complaint(s), previous evaluations and treatments. Parents are mailed an 
application packet consisting of an introductory report form covering parents perception of the child, the 
Family Environment Scale (FES; 25), the Child Behavior Checklist (CBCL; 26), releases of information 
for the school, pediatrician, and previous treatment providers, a questiormaire regarding financial 
information, and a stamped, addressed return envelope. Once the packet is returned, the case is opened 
and assigned to a multidisciplinary team for evaluation. 

Analytic Plan 

We were primarily interested in examining characteristics of African-American and Caucasian patients. 
Thus, subjects were compared using 

chi-square analyses on a number of relevant categorical variables, including gender, age, single vs. 
two-parent home, urban vs. suburban residence, pay status (insurance vs. Title XIX), Hollingshead 
Index, referral source (coercive vs. non-coercive) and presenting problem (internalizing vs. 
externalizing). 

We wished to investigate whether African-Americans and Caucasians differed in likelihood of being 
referred for an externalizing or internalizing problem, of being referred coercively, and in DSM-III-R 
diagnosis, which would be assessed at the end of the evaluation period. To test these hypotheses, we 
again utilized chi-square tests. To examine whether groups differences existed on a parent report 
measure of symptomology, the CBCL, and a therapist measure, the C-GAS, ANOVA's were conducted 
with ethnicity as the independent variable. To understand the relationship of ethnicity to presenting 
problem we utilized log linear analyses to control for gender, SES and coercive referral. 
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Results and Implications 

When we compared African-American and Caucasian patients we found no differences in gender or age. 
However, a significantly higher percentage of African-Americans came from the immediate New Haven 
urban area as compared to Caucasians (80% vs. 17%). In addition, significantly more 
African-Americans came from single parent families (82% vs. 55%) and were on Medicaid (75% vs. 
38%). Compared to Caucasians, a greater number of African-American patients appeared to be 
disproportionally distributed in the lower SES brackets of the Hollingshead (Classes IV, V; 71% vs. 
46%). 

At the time of intake, African-Americans were almost 3 times more likely than Caucasians to be referred 
by coercive sources (61% vs. 22%). Also, African-Americans were significantly more likely than 
Caucasians to be referred for externalizing disorder at intake (88% vs. 67%). Of the children who 
presented with externalizing behavior, 55% of the African-American children were referred by coercive 
sources, compared to 1 5% of the Caucasians. 

When African-American and Caucasian children were compared on the CBCL, more African-American 
parents reported externalizing symptoms (p < .05). However, this difference was no longer significant 
when we controlled for SES. African-American and Caucasian children were not rated differently on the 
internalizing scale. 

On a therapist rated measure of global functioning, the C-GAS ratings for African-Americans and 
Caucasians did not differ significantly. There was no significant difference in the proportion of 
externalizing or internalizing diagnoses between African-Americans and Caucasians. 



Eighty -eight percent of African-Americans were referred for an externalizing problem, 
and 12% for an internalizing problem. At the completion of evaluation, however, 50% 
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received an externalizing diagnosis and 50% an internalizing diagnosis. Similarly, 67% of the 
Caucasians were referred for an externalizing problem, 33% for an internalizing problem; 
at evaluation 36% of this group received externalizing diagnoses, and 64% internalizing diagnoses . 

Log linear analysis was utilized to examine the relationship between ethnicity and reason for referral 
while statistically controlling for SES (I -III vs. IV-IV), referral source (coercive vs. non-coercive) and 
gender, as well as, the interaction of ethnicity and referral reason. Results indicate that once the variance 
attributable to these variables was partialled out, that African-American children were still more likely 
than Caucasians to be referred coercively (Z = 5.08, 12 < .0001), and more likely to be referred for an 
externalizing rather than an internalizing disorder (Z = 3.93, p< .0001). 
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Discussion 

In our study, the African-American children and adolescents were primarily from the city, 
from single parent families, socioeconomically disadvantaged and likely to be coercively referred for an 
externalizing problem. In contrast, Caucasian children and adolescents were likely to be from the 
suburbs, insured, from two parent families and to be non-coercively referred for an evaluation. However, 
their diagnoses indicated similar levels of psychiatric impairment. In our analysis this held when we 
controlled for gender and SES for the African-American group. The findings in our clinic corroborate 
those of other studies. Compared to Caucasians, African-American children, despite a similar clinical 
profile were at greater risk for coercive referral and for externalizing presenting problems. 

One speculates, with other authors, about the perception of African-Americans by referring agencies. 
Fabrega and colleagues (1993) observe that "social systems appear" less tolerant of AfricanAmericans 
and hence their group may be "selected for," in that the social system compels them to obtain (not select) 
care (p. 412). This interpretation is supported by the finding discussed above. These authors question the 
assumption that minorities avoid seeking psychiatric help, resulting in delay in obtaining needed service. 
They suggest the opposite&shyp;based on their fmdings&shyp;that AfricaaAmericans may be "shunted" 
to psychiatric service earlier than Caucasians as a result of more surveillance and discomfort on the part 
of social agencies. Takeuchi and colleagues (1993) suggest a labeling perception on the part of social 
service agencies. However, they identify poverty, rather than race, as the identifying characteristic which 
elicits labeling. Since most of the African-American children seen are poor, there is more likelihood that 
they may be referred by the myriad social service agencies with which they have contact. Although our 
findings point to similar conclusions, our results were not altered when we controlled for SES. Hence, 
this tentatively suggests a more general referral bias that goes beyond socioeconomic disadvantage. 

Of the possible explanations for our findings, referral bias seemed the most plausible given the results of 
this study and previous research; however, the role of ethnicity and race in psychopathology is complex 
and poorly understood, and this conclusion is tentative. Further research controlling for factors such as 
socioeconomic status is needed to examine relationships between ethnicity and psychopathology and the 
presence of bias in mental health services. 
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We are all struggling with systems change and how to create knowledge development strategies that will 
inform our efforts to develop better systems of care for children and families. It is important, now more 
than ever, that we learn from each other in terms of reinventing evaluation. I propose that we follow a 
model of co-leaming &shyp; the co-leamers in this instance being those who are trying to effect change 
and those of us who are evaluating them, or as I like to think, informing them as they do their work and 
develop reflective practice. I would like to make three observations about co-leaming. 

First, with this "co-leaming model" comes a different relationship with the people we serve; it is no 
longer an expert and a subject, or the researcher and the researched. We cannot underestimate the 
difficulty of becoming co-leamers, or the change that it means for all of us, at both ends of the co-leamer 
relationship. 

Second, we need to reinvent evaluation, or knowledge development, so it becomes an internal part of 
doing business in our large systems. Evaluation needs to be part of the process to illuminate support and 
assess our actions as we craft our services for children and families. When it is internalized, it is no 
longer out there being done to us, but instead is in here being done with us. 

Third, we have to recognize the partner who has always been there, and that is the public. We are in an 
era, which I think of as "the new social policy," in which public accountability has come to the forefront. 
So, another constituency or stakeholder in the co-leaming process is the public, and we need to reflect on 
the "public will" that we must generate to maintain the service stmcture, redesigned or not. 

My colleagues and I are learning from actual evaluations of systems change and from the risk takers who 
are trying different evaluations strategies. In addition, we just started a new inquiry for understanding 
process studies. The convention is that you cannot undert^e an evaluation without doing a process 
study. So we have been using our evaluation databases of both comprehensive programs and system 
reform efforts to understand what people mean when they talk about process evaluation. 
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Creating Systems Change 

We have identified some of the earmarks of systems change, the contours of emerging practice, and the 
challenges that come with efforts to do evaluations differently. I want to argue that we cannot do 
systems change unless we also build in the knowledge development strategies that inform change, as 
well as assess it. 

In the child and family arena, we have said that simple programs and categorical approaches are not 
enough and that we need systems change. We have said that we have to develop systems that provide a 
set of services, preventive and otherwise, to children and families. So across the board, we bought the 
notion of systems reform, in part because we know what we have does not work &shyp; not because we 
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know what the shape of the system reform looks like. 

That puts us in a particularly difficult position because we are inventing the system as we implement it. 
Systems change involves many actors and reconfiguring so many different things, that we are talking 
about creating a very, very complex entity. 

We know that our comprehensive programs have not worked, not least because we caimot scale them up. 
We have spent the last ten years watching the family support arena evolve from a fairly simple set of 
programs into more comprehensive systems. This has led to a growing realization the family support 
initiatives are context-dependent, that comprehensive efforts require linkage to other services families 
need, and that no single agency can adequately honor a holistic approach to families. We know we have 
to build linkages, collaboration, and coordination. 

Our landscape, however, is full of programs and some systems change initiatives, that demonstrated, 
researched, and died. The challenged, then, is to sustain and scale up these initiatives &shyp; something 
we do not yet know how to do. These efforts bring us instantly into a political context that has to be 
honored-again, this comes back to the issue of "public will." We must create knowledge and 
accountability that will warrant public faith in what we are trying to do. 
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The Need to Reinvent Evaluation 

We can start by building our case on evaluations of CASSP and, other, usually single system, reform 
initiatives that suggest systems reform is possible. Currently, we are at the same point in evaluation as 
we are in systems reform. We need to recreate and rethink what we do, but it is not clear what the path 
ahead looks like. Change will require a willingness to take risks, and feedback through a co-leaming 
model so we can revise our evaluation practices, as well as our program practices. Hence, we have to 
reinvent evaluation and knowledge development, so that they become features of systems reform that 
work hand-in-hand with the effort. 

Systems reform is difficult, and requires much of evaluators. The goals of system reform include the 
major restructuring of existing services. We need to rethink and redirect public resources and public 
services into a coherent system. This cross-system effort should look at the needs of an entire 
community, not simply particular population groups. It also should involve service strategies that are 
different, those that go beyond the coordination of existing services and create a new system that is 
integrated and user-friendly. It should be a service system that operates differently, and looks and feels 
different to families who are using the system. We must involve multiple stakeholders in these efforts, 
including program participants, front-line workers, managers, and those who decide how resources are 
allocated in communities. 



Additionally, we need to build accountability into the resource allocation process. This means creating 
responsive governance entities that can manage accountability and design different financing strategies. 
We cannot underestimate the risk, complexity, and challenge that accountability poses with multiple 
stakeholders at the state and local level in systems reform. 

We need to demonstrate that systems reform is productive, that it can be implemented, that it produces 
important outcomes, and that it is better than alternative approaches to organizing and delivering 
services. With this comes a need for internal validity, high quality evidence, and external validity, so that 
this evidence is useful not only to evaluators, but to other stakeholders as well. We need to inform 
program development, build and sustain the public vsdll, and enlist a high degree of participation to make 
the overall system work. 



These are enormous challenges to address, particularly in the context of the new social policy. Under the 
new Congress we are going to have more and more decentralization, more and more block granting, and 
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fewer and fewer entitlements. We are going to have a very different look in states and communities 
around the country. There are new players and new stresses. I think there will be incredible competition 
for increasingly scarce resources. 

Return to 1'op 



What Will Change Mean? 

What does this mean for those of us who have been laboring in the vineyards of systems reform? I think 
it means we have to redouble our efforts and confront a number of challenges. A couple of years ago the 
governor of West Virginia set up a new cabinet for children, youth, and families that aimed to 
decentralize planning and decision making about the use of resources for children and families in the 
state. The cabinet gave money to a number of communities around the state and asked an array of 
players to participate. They were told that they would have control of much of the existing revenue, and 
they had to plan a new system which met everybody's needs, underscored prevention, and also made it 
possible to access the kind of treatment that some families would need. 

These communities had to plan, implement, and assess the effectiveness of their efforts. They were 
scared to death and very much felt they did not have the kind of information they needed. They went out 
and collected some demographic data, but realized that it was necessary, but not sufficient. They then 
started doing surveys, focus groups, and so forth, to get the input they needed for planning this new 
system of services. When it was time to evaluate their efforts, they were terrified. They could not find an 
evaluator who would come in, especially with their very scarce resources, to help them perform a 
participatory evaluation. 

This, I think, is the reality in many, many communities. We decentralize and say that we are 
empowering communities, but we give them disempowering evaluations and fail to make them 
stakeholders in the entire process. I think we need to come to grips with this as we reinvent evaluations. 

What are some of the contours of the new evaluation practice? We need to move beyond experimental 
research design. Much of our practice, particularly as it has been played out at the highest levels of 
government, says that there is no knowledge past random assignment. We need to build a case for 
developing other methods that are stringent and that provide useful knowledge, above and beyond what 
can be achieved through an experimental research design. There are questions about the ability to 
implement, pure and simple, an experimental research design in systems reform efforts. Usually when 
one does experimental research design, it is with a program. 

There is a specified treatment, an external evaluator. From this we determine whether something works 
or not&shyp;usually not "how" something works. In experimental research designs, we make a whole 
set of assumptions about research validity and "treatment," which I think are not possible to guarantee in 
systems reform efforts. 

As I have looked across the terrain in the 20 years that I have studied programs for children and families, 
I have not seen any examples in which a well-done evaluation, meeting the conventions of experimental 
research design, has resulted in any kind of significant program expansion. I have increasingly come to 
think that, in the social services arena, these evaluations create reasons why we don't do things, as well 
as inform what we do. 



We need to ask what we need to prove, to make a plausible case, that some new initiative is working. 
We need to identify alternatives to experimental research design, and ask the question at the community 
and state level, "who are the stakeholders, what is proof, and what's a plausible case that says that 
systems reform works?" 



People like me argue that we need alternatives to conventional practice. In fairness, I am asking you to 
open up a Pandora's box of problems. The more we look at alternative practices, the more I see comfort 
in the traditional recipe for doing conventional research using experimental design with an experimental 
evaluator who is the expert, assumes control, and so on. What does evaluation as a helping relationship 
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really look like? It means that the role of the evaluator is fundamentally different, and it means a lot of 
stress and tension as we try to redefine this helping relationship &shyp; the co-leaming model, if you 
will. It means we are in a context with different stakeholders who have different expectations for what 
we are going to do. Part of the co-leaming model is thinking through those different expectations and 
somehow working out a reasonable way of proceeding in the face of them. 

I know it is not "going native" as they say in anthropology, and becoming completely the voice of the 
program. I know it is not being remote and disconnected. The answer lies out in the middle, in 
unexplored territory, particularly against the backdrop of a group of stakeholders who have different 
ideas about objectivity, bias, and the rest of it. 

Return to Top 



Building a System of Accountability 

What does it mean to build a system of accountability? It is very different than building a good 
evaluation and knowledge development strategy. Accountability is when you learn and see how you are 
doing. You also generate stakeholder involvement so there will be an audience who will listen to what 
you have found, who will then have the responsibility for problem solving, and when necessary, pulling 
the plug when something is not working. This is a different game than evaluators have been in before. 
There is an emphasis on problem solving and midcourse correction, but there also the notion of "plug 
pulling" that we may not have faced in the way we will need to in this new social policy environment. It 
also means figuring out the rewards for success for risk takers. 

We also need to understand the process of developing systems change initiatives. Some of the work by 
the late Judy Gardner and her colleagues at the Heller School at Brandeis suggests that systems reform 
efforts build on one another &shyp; that CASSP created the infrastructure on which the Robert Wood 
Johnson Mental Health Services for Youth Initiative was able to build. We need to get the historic 
dimension, and get a sense of the infrastructure of systems reform, document that, and extract the 
lessons for others. One of those lessons may be that systems change takes time. 

With this new understanding of process comes the notion of mixed methods, and getting beyond what 
we call anecdotes for "correlation coefficients." We've been looking at process studies and are amazed at 
how many of the process studies embedded within evaluations are about "parallel play." The correlation 
coefficient is over here, and the case study that demonstrates that is over there. There is really a need to 
connect these two methods so they work together, complement one another, and produce knowledge that 
cannot be generated from a purely quantitative or a purely qualitative strategy. 

We need to spend more time in training and capacity building for evaluators. One of the things we hear 
is how few evaluators there are who have the negotiation and other skills necessary to work in these new 
ways, in this co-leaming model. As evaluators, we need to examine how we are training people in these 
skills that will be requisite, and how we can get this training in place. It may require developing 
apprenticeships with innovative evaluators that enable the next generation to learn from the experience 
and the stmggles of this first generation of risk takers. 

There are a set of earmarks for this new framework for evaluation. First, this new strategy must develop 
the internal capacity to collect, analyze, interpret and utilize a wide variety of data. It must incorporate 
knowledge development as one of the central functions of systems reform itself 

Secondly, the work is never done. When we develop a system, it has to be one that is continually 
reinventing itself as circumstances change and as the needs of children and families change. This 
capacity for continuous self-reflection an important element. We must use information to inform a wide 
variety of stakeholders, and build consensus about the outcomes we want and the strategies we will use 
to improve those outcomes. Finally, we must use information to build and enforce a sense of public 
accountability for this process. 

When we looked at examples of people who have been trying to do these strategies, we found four types: 
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• Mixed methods strategies, in which people are using quantitative data and simultaneously talking 
to communities and trying to get participant perspectives on implementation and the value of 
programs; 

• Knowledge development and self-evaluation strategies, which Bob Illback has done in Kentucky, 
and which Lynn Usher has done in the family resource centers in Maryland, as part of the 
Maryland Service Reform Initiatives; 

• Public accountability, which is the outcome-based accountability strand, which you see a great 
deal of, particularly in the United Way Success by Six initiatives; and finally, 

• More participatory strategies which are more in line with some of things I've been suggesting. 

We have learned several lessons from trying these new approaches. One is that it is incredibly difficult 
to promote organizational learning and the capacity for reflective practice. Those difficulties lie both in 
our stance as evaluators and in the receptivity and structure of the organizations we're asking to become 
learner organizations. There's a recent article in Evaluation Practice that talks about how few instances 
there are where organizations have really learned from evaluation. A profound change in organizational 
culture is necessary if we are going to implement these new evaluation approaches. 

We've also found that it is a real struggle getting trained evaluators for this organizational development 
approach to evaluation. They are simply not there, and when they are there, they require time and 
resources, including money, to implement this kind of practice. More self-evaluative, participatory 
approaches mean time on the ground, working with front line workers and managers who are collecting 
data, asking them about how they use that information to inform practice, to inform the multiple 
stakeholders in the community, and to be accountable. We cannot underestimate the time and resources 
that are necessary to make this work. 

I would argue, however, that if we do not open the Pandora's box of these new frameworks for practice, 
we will be stuck in a situation where there are fewer and fewer resources and more public distrust of our 
efforts. Our evaluation project at Harvard Family Research Project is trying to create a broader forum for 
our risk takers. We have launched our newsletter. The Evaluation Exchange, which is meant to be a 
place where people who are trying new approaches can get some visibility for their work and get 
feedback from what we hope will be a growing leadership. We are embarking together down this road of 
trying new strategies, and we need to be co-leamers in sessions such as this, confronting challenges of 
inventing and implementing this new of practice. 

Return to fop 



Heather B. Weiss, Ed.D. 

Project Director 

Harvard Family Research Project 
Harvard Graduate School of Education 
Longfellow Hall, Appian Way 
Cambridge, MA 02138 
617/495-9108 Fax: 717/495-8594 
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Today I am going to discuss with you three themes in the current move to outcome-based accountability 
in systems reform: The first is the current political demand for improved outcomes; the second is the 
way we need to change our responses if we are going to improve outcomes, and third, is the way we 
must set parameters upon which to judge our efforts. 

The Demand for Improved Outcomes 



In today's political climate, policy makers have discovered outcomes and they want to use them on us. 
There is increasing recognition that too many children and families are not succeeding and growing 
rhetorical acceptance that the characteristics of the current service systems&shyp;categorical, reactive, 
fragmented, arbitrary, hierarchical, deficit based, coercive &shyp;do not provide a good match between 
what families need to succeed and what systems offer. 

The growing use of today's new service mantra&shyp;"family-centered, neighborhood-based, 
community-designed, preventive, partnering, flexible, comprehensive, asset oriented, 
voluntary"&shyp;suggests how services have to change to provide a better match and what is needed to 
succeed. 

Belief in this new service mantra requires new accountability that is not based upon adherence to 
procedures but rather upon success in attaining results. 

Increasingly, outcomes are seen as an integral part, and in many respects, a driver of system reform. 
System reform can be conceptualized as requiring integration of new forms of governance, new 
approaches to services, new financing systems, attention to leadership and professional development, 
and clear enumeration of outcomes and means to hold systems accountable to meeting them. 

At the state and community elected official policy making level, officials want something "that works" 
to reduce teen pregnancy, infant mortality and low birthweight, school unreadiness, crime and juvenile 
delinquency, and low slalls, school dropout, and idleness. These officials are not necessarily naive, but 
they want to put the pressure on bureaucracies and service providers for solutions and for fixing what 
advocates and reformers say is broken. "You want flexibility, we want results." 

Return to Top 



Changing our Response 

If we are going to improve outcomes (and to succeed with children and families current systems will 
fail), we need to change our responses at several levels. I've proposed a conceptual framework to view 
needed comprehensive service reforms. 
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The "simple model" depicted in Figure I g enerally produces a constellation of outcomes that are 
multi-dimensional, spanning health, developmental, social, and psychological dimensions. 

The charge to implementors is to improve all these outcomes at a community- wide level. This requires 
change, particularly as it is related to eliminating the categorical deficits of the old system and replacing 
them with the service mantra of the new. 

Some of the positive outcomes for children, which are generally consistent with what community or 
state stakeholders want to produce include: 

• Success in school 

• Responsible sexuality 

• Social consciousness 

• Community involvement 

• Healthy lifestyle 

• Readiness for adulthood 

Policy makers often stop here and turn the responsibilty for improved outcomes over to implementors. 
These can be considered an interrelated constellation not amenable to fragmented, categorical responses. 
These outcome indicators are adapted from Lisbeth Schorr's "minimalist list." 

What we need to ask is: 'Can these long-term outcomes (the British use the term "final" outcomes) be 
used to guide reform?' 'Can they be used to hold systems accountable?' 'Can they inform how different 
aspects of our needed and complicated array of services and supports must change? 

The answer is yes and no. They can be used as a guide to creating pressure for reform. At the same time, 
we need different, intermediary or proximate, levels of outcomes or outputs to use for accountability and 
course correction purposes, applicable in different ways to different parts of the system of community 
supports and services. 

I've proposed a formulation of three conceptually distinct, but interrelated, outcomes or outputs needed 
to expect changes in those long-term goals: the whole village approach, effective frontline strategies, and 
productive service systems. I will describe the importance of each, and then touch on ways that one can 
measure them. 

Whole Villages. The most common frontispiece to a task force on systems change through 
comprehensive services is the African proverb, "It takes a whole village to raise a child." Note that it 
does not say that "it takes a multidisciplinary team with integrated case management to raise a child." 

In distressed and disinvested neighborhoods, however, there often is a threadbare community fabric 
offering little support and opportunity to help families grow and develop. The challenge raised by the 
proverb is really, "How do you make whole villages, or make existing villages whole?" 

There exists substantial empirical literature that living in distressed and disinvested neighborhoods 
constitutes a major risk factor to healthy growth and development. Various formulations incorporating 
these village factors into reform strategies have been offered: Cataldo and Hawkins and others have 
marketed them as "protective factors;" Chapin Hall speaks of them as "primary services;" Putnam and 
others describe this as "social capital;" and Public/Private Ventures speak to them as "core concepts." 

One enumeration of these community elements or factors in child success can be described as: 

• Social and economic stability, 

• Safe and stable neighborhoods, 

• Nurturing and mentoring adults, 

• Positive peer attitudes, 

• Diversity of "social capital" &shyp; school, church, neighborhood, civic activities, and 

• Opportunities to realize aspirations. 

Does this list make sense? Clearly, where these do not exist in a neighborhood (whatever other services 
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are provided), children are not likely to succeed. Where they do exist, most children do pretty well. 

Developing strategies to construct such core elements represents a challenge that must be met if the final 
outcomes are to be achieved. We're in our infancy of exploring how to do that. At the policy level, recent 
proposals to offer Midnight basketball to deter juvenile crime is a good illustration. 

While we may not have answers on how government can be a catalyst for the creation of "Caring 
communities", we can develop metrics that do a reasonable job of measuring each of these conditions. 
Some approaches, most notably family support centers and settlement houses, seek to construct this 
village as well as work with individual families. There are outcome measures or indicators that can be 
tracked to determine whether communities are rebuilding, as this more detailed iteration of whole village 
concept for to child and youth suggests. 

Effective frontline strategies. The second change that needs to occur is in frontline service delivery. 
How do we measure, and hold accountable, this new service mantra frontline approach that is 
individualized in its response to families, given that families are messy units of analysis and do not fit 
comfortably within regression equations? 

First, we should have process measures to determine whether services actually are community-based, 
individually-tailored, asset-oriented, and comprehensive, which in itself is a major task. I talked with a 
collaborative leader enamored of Lisbeth Schorr's book. Within Our Reach, whose solution was to beat 
workers over the head with it until they agreed to practice exactly what she said. This service mantra is 
easier to enact into policy and agency mission than it is to create in practice. 

With outcomes, however, I believe we must develop means to determine whether individual workers are 
making progress with families. Without this, broader accountability will be impossible&shyp;with it, 
that accountability may be complex but is at least technically possible. The following characteristics of 
effective front line interventions adapted from my paper in Ae National Center for Service Intergration 
Brief, Making a Difference, suggests different levels of examining such service approach outcomes at 
the frontline level: 

• Identify Appropriate Families (Service Penetration), 

• Engage Families, 

• Aid in Family Growth, 

• Connect to Community-Building, and 

• Produce Changes in Other Systems. 

This can provide a discipline for programs involved in this admittedly "mushy" family support/frontline 
practice. This is critical to do, because the initial gains with families may be quite removed from 
long-term community outcomes. 

For example, let me illustrate some of the steps families in Iowa's Way to Grow program took and 
contrast them with the community outcomes that funders attached to the program. This comparison 
indicates that the relationship between what steps families take and what long term outcomes policy 
makers seek usually is not linear or direct (See Table 11. 

Productive Service Systems. The third issue in developing positive outcomes addresses systemic 
responses-&shyp;raising issues of collaboration, referral, and integrated activities. We still need 
expertise and professional service responses-&shyp;but we need them to practice differently and connect 
better with one another. 

Suggested outcomes we want to see from changes in the manner in which our systems respond to 
families might include: 

• Seamless and coordinated response, 

• Integrated into community life, 

• Timely response to opportunities. 
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• Sufficient to meet objectives, 

• Efficient in meeting objectives, and 

• Adaptive and learning objectives. 

Again, these are based upon common sense notions of where our current systems often fail to respond 
adequately and appropriately. They are based upon what we believe, from both theoretical and clinical 
roots, systems need to do to respond productively to children and families. 

These system outcomes or outputs can be measured as well&shyp;often even at a performance tracking 
and management information system level. In child welfare, for instance, the implications of such an 
approach to permanency planning would be that a premium would be placed on more immediate 
response, minimizing disruption and retaining connected with family the these when placement occurs, 
and establishing the expectations for everyone in the system to achieve permanency. This is in dramatic 
contrast to the usual business in child welfare of placing a child, breathing a sigh of relief, and beginning 
the process of assessing reunification potential only as the court review date approaches. 

There is nothing terribly profound in all this. It seems to me that positive final community outcomes 
logically must be dependent upon whole villages, effective frontline approaches, and productive service 
systems. The absence of any diminishes the ability of all to reach those outcomes. Think of it as basic 
common sense. In fact, our Iowa Kids Count book is entitled "Reinventing Common Sense" to signify 
how remote our systems often seem from basic principles of how children and families succeed. 

Return to Top 



Setting Parameters 

If we want outcomes to be our friend, we must convincingly set out the parameters upon which our work 
should be judged. 

What I talked about is not the answer that elected officials want to hear. It is not a single program or 
service that lends itself to sound bites or produces impressive results before the next election. At the 
same time, however, although elected officials live in the electoral world, they also know that these are 
tough issues and they want solutions, even if they are complex and long-term. What they need is clear 
talk on what to expect and what it will take. 

We must make a case for "scale." Strategies must be adopted at a scale sufficient to influence outcomes 
on a community-wide basis. Returning to the conceptual framework I presented earlier, we must make 
the case that sufficient resources must be devoted to the strategies that, if they work, they will succeed 
with a large enough number of families and children to produce change in long-term community 
outcomes. 



We must not leave parts of the puzzle out of the picture. We must be clear of what we can, and cannot 
do, particularly if we are not going to address community disinvestment and seek to make villages 
whole. We will experience intervention "fade-out" if the context in which families take steps remain 
hostile to future gain. 



We must place appropriate expectations regarding timeframes. The Perry Pre-School Project has been 
widely regarded by policy makers as sound evidence that high quality pre-school programs are 
cost-effective. Yet our return on investment is a decade-and-a- half down the road. Patience is needed, if 
generational effects take a generation to occur. We must create realistic political expectations; you 
cannot force generational effects into a two-year election cycle window. 



We must present issues of cost in context . In Michigan over the last two decades, spending on housing 
prisoners has increased from 3 % to 14 % of state spending&shyp; approximately $1 billion increased 
share of the total budget. That would ftmd a lot of family support and community-building services 
which might have done far more to protect the safety and well-being of Michigan residents than housing 
35,000 more prisoners. 
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Ultimately, we must change our own efforts when they do not produce the intermediate or proximate 
outcomes that we believe they need to produce to achieve success. We do need to learn by doing, and if 
we take risks we will have failures. The challenge is not to institutionalize them but to learn from them 
to eventually succeed. 

Return to Top 
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Charles Bruner serves as Executive Director of the Child and Family Policy Center, a nonprofit 
organization established in 1989 "to better link research and policy on issues vital to children and 
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Through the Child and Family Policy Center, Bruner provides technical assistance to states, 
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to the Annie E. Casey Foundation in developing its Family-to-Family Initiative, its strategies for 
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Bruner has written widely on public policy approaches to developing more comprehensive, 
community-based responses to children, family, and neighborhood needs. 

Charles Brunner 

Executive Director 

Child and Family Policy Center 

100 Court Ave, Suite 312 

DesMoines, I A 50309 
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Mary Telesford, Federation of Families for Children's Mental Health, Washington, 
DC. 



After hearing Dr. Bruner's remarks, I also like to re-emphasize the need for strength-based models of 
service delivery. Too long we have emphasized the deficits of our communities which so often 
reinforces the negative outcomes we've come to expect. 

Let me tell you about some of the strengths that impoverished communities do have, and that researchers 
need to recognize and work with. 

1. Churches are community institutions that often know the history of what's happened in a 
neighborhood and have the ability to organize constituencies; 

2. Community-based organizations also have a tie to a community's history, proximity to the people 
and knowledge of culturally relevant issues; and 

3. Schools can offer access to many communities and families and can provide a neutral meeting 
ground. 

From my recent work (unclear reference) we've been able to observe the following pattern among 
women of color who live in impoverished circumstances and are successful. 

Foremost, they provide a spiritual base for their families and maintain a strong support system. They 
also maintain personal sobriety and do not use alcohol or illegal substances. They have meaningful 
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employment and value education, even if it has been delayed or interrupted. 

Clearly if we want to build successful communities, we need to build upon strength-based models and 
reach out to the people and families we serve. 

We need to understand what families want, which is: 

1 . to receive support when they need it, 

2. to have a single place for services&shyp;a one-stop shop, instead of the myriad service systems 
they face; and 

3. to receive recognition for their efforts. 

If we can help families and recognize their strengths, then the outcomes will take care of themselves. 

Cliff Davis 

Chief, Office of Children's Services and Prevention, Ohio Department of Mental Health 

I'd like to add, within the context that Dr. Bruner described, one piece that is important to our 
understanding outcomes achieved from the helping process. 

Regardless of the theories you have been trained in or the therapeutic tools you use with the persons who 
come to you for help, the only thing we know for sure is that the quality of the relationship between the 
person receiving or seeking help and the person offering help is a key to what kinds of outcomes are 
achieved. 

I am a therapist by trade, have spent much of my career as a community-based family therapist, and I 
know I did many good things&shyp;I know intuitively, instinctively&shyp;my wife could tell by 
looking at me whether the work I had done that day felt good to me. 

Regardless of the type of indicators we talk about, we are still a long way from being able to define or 
articulate this important relationship quality for people who are involved in the helping process. Until 
we are able to work together to understand, articulate, and measure (if we have to), that relationship and 
its components, we still aren't really going to understand what it is that we need to be training, and what 
it is we have to be doing in order to achieve positive, successful outcomes for our kids and their families. 

As we build processes to try to measure, understand, and articulate how we are affecting children and 
their families, we have to start with the families and front line workers. We have to work hand-in-hand 
with them throughout that process and ultimately the measure of our success will be determined by those 
families and front line workers. 

It does no good for me to sit in the state office tower in downtown Columbus, in central Ohio, and 
establish outcome measures that ultimately determine whether or not my system is successful. It has to 
start through focus group discussion, and implementation of participatory processes with families and 
front line workers; they have to be integrally related throughout the whole process and it is up to them to 
interpret the results. We need to create the mechanisms that allow this to happen. 

In the context of all the different community factors that Dr. Bruner described, asking families and 
workers to help us strikes me as one of the keys, at least in the mental health system, of the work that we 
are doing. I had compiled a list of real life kinds of indicators&shyp;I suspect everyone in this room 
could generate a similar kind of list. We need to pay attention to those things that our families, our kids 
and our front line workers say are the true indication of what our system is doing. 



About the Lecture Series 



The Gwen R. Iding Brogden Lecture Series was established as a lasting tribute to a citizen advocate. For 
more than 20 years, Ms. Iding Brogden championed the creation of high quality mental health services 
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in local, state and national arenas. The Lecture series features national speakers known for significant 
contributions in the area of children's mental health policy. For more information, contact Larry 
Schwartz at the Department of Child and Family Studies, Florida Mental Health Institute, University of 
South Florida, 813/974-4641. 
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Table 2 

’’Whole Village” Keys to Child, Youth Success - 
Existence of Public Goods and Opportunities 

Stable environment for families, socially and economically 

• adequacy of housing stock 

• reasonableness of rent 

• mobility rates 

• employment opportunities 

• availability of general public goods (transportation, retail goods, recreation, etc.) 

Safe neighborhood and community 

• crime 

• environmental conditions (lead paint, abandoned buildings, etc.) 

Nurturing relationships with adults 

• parental participation in school life and activities adult participation in developing and supporting 
child and youth activities 

• places knovm by community as supportive learning environments (e.g. family centers) 

• array and supply of quality child care arrangements 

• schools as safe and supportive places for children 

Involvement in positive peer activities 

• participation levels in youth recreational activities, numbers and diversity 

• supervised play areas, parks, and events 

Diverse connections with community &shyp;schooI, church, neighborhood, civic activities 

• public library usage 

• church membership and activities 

• youth-based organi 2 ations 

• community recognition of youth in events and activities 

• voter registration and participation 

Realistic opportunities to realize! aspirations 

• business sponsorship of employment-education connections 

• diverse choices to excel and succeed in and out of school 

• positive climate and attitude among residents toward achieving potential 
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